-- Logs begin at Fri 2022-07-15 14:48:47 UTC, end at Fri 2022-07-15 16:20:24 UTC. -- Jul 15 15:31:24.161372 dv-cinder-835878-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 15 15:31:24.170745 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 15 15:31:24 2022] *** Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: nodename: dv-cinder-835878-3-iscsi Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: machine: x86_64 Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: clock source: unix Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: pcre jit disabled Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: detected number of CPU cores: 4 Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: current working directory: / Jul 15 15:31:24.172792 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: detected binary path: /usr/bin/uwsgi-core Jul 15 15:31:24.173319 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: your processes number limit is 47680 Jul 15 15:31:24.173319 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: your memory page size is 4096 bytes Jul 15 15:31:24.173319 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: detected max file descriptor number: 2048 Jul 15 15:31:24.173319 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: lock engine: pthread robust mutexes Jul 15 15:31:24.173319 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: thunder lock: enabled Jul 15 15:31:24.173319 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 15 15:31:24.173319 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 15 15:31:24.213363 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: Python main interpreter initialized at 0x557467d489c0 Jul 15 15:31:24.213363 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: python threads support enabled Jul 15 15:31:24.213363 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: your server socket listen backlog is limited to 100 connections Jul 15 15:31:24.213363 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: your mercy for graceful operations on workers is 90 seconds Jul 15 15:31:24.213710 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: mapped 403077 bytes (393 KB) for 2 cores Jul 15 15:31:24.213785 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: *** Operational MODE: preforking *** Jul 15 15:31:24.214046 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: *** uWSGI is running in multiple interpreter mode *** Jul 15 15:31:24.214046 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: spawned uWSGI master process (pid: 87012) Jul 15 15:31:24.214245 dv-cinder-835878-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 15 15:31:24.215084 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: spawned uWSGI worker 1 (pid: 87025, cores: 1) Jul 15 15:31:24.215174 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: spawned uWSGI worker 2 (pid: 87026, cores: 1) Jul 15 15:31:24.215894 dv-cinder-835878-3-iscsi devstack@keystone.service[87012]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 15 15:31:29.721119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 15:31:29.721119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: from cryptography.utils import int_from_bytes Jul 15 15:31:29.726728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 15:31:29.726728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: from cryptography.utils import int_from_bytes Jul 15 15:31:29.726864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 15:31:29.726864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: from cryptography.utils import int_from_bytes Jul 15 15:31:29.726864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 15:31:29.726864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: from cryptography.utils import int_from_bytes Jul 15 15:31:29.770434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 15:31:29.770631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 15:31:29.770826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] command line args: [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 15:31:29.771100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 15:31:29.772097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ================================================================================ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 15:31:29.772233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 15:31:29.772350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] admin_token = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.772438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 15:31:29.772598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] config_dir = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.772686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] command line args: [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 15:31:29.772788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.772880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 15:31:29.772958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] config_source = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.773035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ================================================================================ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 15:31:29.773141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] debug = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.773286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] admin_token = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.773371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.773663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] config_dir = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.773775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.773879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.773982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] config_source = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] debug = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.774939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_config_append = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_dir = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_file = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_config_append = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.775918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_dir = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_file = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.776954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.777994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.778114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.778194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.778273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.778389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.778505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.778652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.778775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.778983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.779144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.98/identity {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.779231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.779350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] publish_errors = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.779535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.779626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.779807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.779988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.98/identity {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] publish_errors = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] standard_threads = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.780939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_journal = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] standard_threads = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_json = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.781952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_stderr = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_syslog = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_journal = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.782559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_json = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_stderr = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.782893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_syslog = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.782976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.783130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:31:29.783216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.783470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.783561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.783716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.783795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.783967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.784067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.784208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.784287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.784459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.784543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.784704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.784787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.784954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.785037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.785192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.785271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.785440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.785518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.785678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.785755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.785928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.786013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.786165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.786247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.786412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.786498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.786683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.786801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.787463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.787592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.793994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.795989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.796981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.797952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.798977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.799925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.800972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.801931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.802944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.803953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804030 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.804928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.805943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.806899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.807044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.807131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.807310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.807393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.807559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.807650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.807821 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.807907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.808116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.808194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.808377 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.808464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.808616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.external = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.808700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.external = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.808854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.808938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.809162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.809282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.809398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.809487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.809631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.password = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.809711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.password = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.809865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.token = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.809943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.token = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.810109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.810196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.810348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.810435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.810599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.810706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.810837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.810949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.811091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.811205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.811369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.811489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.811610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.811731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.811867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.811975 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.812093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.812230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.812348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.812486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.812603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.812742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.812848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.812987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.813105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.813238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.813347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.813481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.813620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.813752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.813921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.814075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.814157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.814231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.814356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.814492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.814645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.814781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.814897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.815034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.815163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.815340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.815424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.815810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.815906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.815981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816295 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816548 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.816888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.817037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.817116 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.817290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.817370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.817540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.817630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.817786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.817864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.818028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.818106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.818305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.818407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.818550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.818629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.818787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.818875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.819034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.819120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.819283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.819381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.819544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.821949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.822898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.823985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.824956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.825968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.826893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.827057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.827137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.827385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.827478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.827567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.827643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.827797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.827878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.828041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.828120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.828287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.828369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.828528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.828607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.828768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.828846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.829978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.830064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.830216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.830295 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.830457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.830538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.830696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.830774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.830928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.831007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.831160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.831240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.831429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.831512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.831667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.831745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.831918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.832966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.833120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.833198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.833356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.833436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.833592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.833671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.834326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.834406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.834561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.834641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.834814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.834893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.835059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.835138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.835353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.835438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.835577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.835657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.835811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.835890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.836046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.836126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.836289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.836367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.836530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.836609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.836768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.836848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.837980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.838058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.838219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.838298 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.838464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.838536 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.838708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.838801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.838942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.839024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.839182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.839262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.839448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.839528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.839691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.839773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.839926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.840984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.841147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.841229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.841389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.841468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.841650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.841737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.841891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.841972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.842131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.842210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.842371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.842508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.842614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.842694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.842853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] role.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.842934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.843091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] role.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.843178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] role.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.843347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.843431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] role.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.843595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.843673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.843844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.843936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.844095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.844174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.844342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.844426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.844590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.844677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.844837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.844916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.845085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.845164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.845332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.845418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.845566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.845646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.845805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.845883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.846053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.846141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.846303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.846424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.846584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.846662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.846792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.846945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.847186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.847265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.850841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.850973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851463 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851463 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.851878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852877 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.852948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853380 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.853949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.854967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855188 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855499 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855844 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.855925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.856096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.856215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.856396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.856476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.856661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.856755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.856914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.856995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.857168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.857323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.857399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.857470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.857643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.857723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.857882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.857969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.858119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.858198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.858358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.858436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.858600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.858690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.858837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.858916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.859077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.859156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.859349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.859447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.859582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.859662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.859820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.859899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.860925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 15:31:29.860997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:31:29.861166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 15:31:29.891934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.892167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.892399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.898858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.899099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.899357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.902604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.907743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.929968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.930195 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.933574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:29.933817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.073641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.073818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.122115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.122347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.122464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.122706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.186927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.187156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.187285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.187415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.187549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.187631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.187776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.187857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.187999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.188087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.188219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.188437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.213185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.213391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.213486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.213599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.213682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.213828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.213909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.214055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.214144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.214281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.214400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.214484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.214549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.214705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.226057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.226396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.238726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.238948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.239987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.240135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.240214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.240351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:31:30.257778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.257990 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 15:31:30.258218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.258436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 15:31:30.258553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.258761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 15:31:30.258998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.259082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.259205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 15:31:30.259288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.259508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.259592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.259779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.259859 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.260043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.260306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.270595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.270874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.271115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.271288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.271499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.271603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.272046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.272337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.280876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.281044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.284186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.284356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.284471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.284645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.284762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.284925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.285047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.285209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.285332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.285492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.285595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.285754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.285868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.286024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.286145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.286301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.286571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.286688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.286869 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.286979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.287141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.313784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.314303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.315413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.315699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.315977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.316264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.320982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.321268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.321551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.321632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.321868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.322152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.329001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.329243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.329613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.329903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.333691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.334493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.334776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.334912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.335056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.335202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.335352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.335511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.335631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.335797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.335891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.336078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.336160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.336361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.336633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.336902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.354692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.354971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.355556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.355841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.359494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.360358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.363274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.363594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.366531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.366826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.369250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.372002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.372897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.375612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.376540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.376842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.377130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.377426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.377716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.378012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.379254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.379584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.379876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.380165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.380466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.381333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.400751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.401035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.401134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.401306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.401435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.401572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.401662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.401844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.409507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.409739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.411237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.411483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.411568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.411764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.414740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.414945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.415028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.415246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.415348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.415565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.415650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.415858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.415941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.416146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.429766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.429859 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.430053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.430137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.430338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.430434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.430625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.430709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.430913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.430997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.431193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.431278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.431494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.431577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.451283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.451406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.454563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.454650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.457886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.457992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.459542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.459641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.461320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.461414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.461603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.461698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.461887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.461971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.469147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.469240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.470763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.470860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.474017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.474139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.474329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.474464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.474624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.474750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.474900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.475027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.484108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.484280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.484401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.484572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.484687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.484913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.493041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.493327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.493612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.493892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.494181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.494474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.494750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.495016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.495412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.495577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:31:30.495699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.495867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.495985 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.496156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.496248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.496437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.496582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.496723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.496866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.496995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.497272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.497543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.497913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.498206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.498492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.498776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.499055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.499345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:31:30.535216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557467d489c0 pid: 87026 (default app) Jul 15 15:31:30.538688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f819ccc-d181-4f7e-ae8e-b991412e6e05 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:30.538987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f819ccc-d181-4f7e-ae8e-b991412e6e05 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:30.539270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f819ccc-d181-4f7e-ae8e-b991412e6e05 None None] PATH_INFO: `/v3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:30.539781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557467d489c0 pid: 87025 (default app) Jul 15 15:31:30.543538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1/1] 10.222.0.98 () {50 vars in 791 bytes} [Fri Jul 15 15:31:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:39.248027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a89a-e38b-4f56-b29b-9a80b22a783e None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:39.248682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a89a-e38b-4f56-b29b-9a80b22a783e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:39.248682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9a89a-e38b-4f56-b29b-9a80b22a783e None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:39.252962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1/2] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:39.259020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:39.259482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:39.259660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:39.328507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: INFO dbcounter [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] Registered counter for database keystone Jul 15 15:31:39.349948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG oslo_db.sqlalchemy.engines [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87026) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 15:31:39.357252 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writer thread running {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 15:31:39.535046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG passlib.handlers.bcrypt [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87026) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 15:31:39.535415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG passlib.handlers.bcrypt [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87026) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 15:31:39.590971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:39.592651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:39.715606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 2/3] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 15 15:31:39.758348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9a89a-e38b-4f56-b29b-9a80b22a783e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:39.829818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: INFO dbcounter [None req-49e9a89a-e38b-4f56-b29b-9a80b22a783e None None] Registered counter for database keystone Jul 15 15:31:39.849155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG oslo_db.sqlalchemy.engines [None req-49e9a89a-e38b-4f56-b29b-9a80b22a783e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87025) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 15:31:39.855857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writer thread running {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 15:31:39.975813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9a89a-e38b-4f56-b29b-9a80b22a783e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:39.980176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e419e-9aee-4c19-a980-2e18dd9e01cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:39.980852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e419e-9aee-4c19-a980-2e18dd9e01cb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:39.981142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e419e-9aee-4c19-a980-2e18dd9e01cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:39.981425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e419e-9aee-4c19-a980-2e18dd9e01cb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:39.989972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e419e-9aee-4c19-a980-2e18dd9e01cb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:40.042684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9e8e419e-9aee-4c19-a980-2e18dd9e01cb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 15:31:40.043023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e419e-9aee-4c19-a980-2e18dd9e01cb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:40.053102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 2/4] 10.222.0.98 () {58 vars in 1324 bytes} [Fri Jul 15 15:31:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:31:40.059168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:40.074096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a41d20-7c1b-4a2d-926b-f0d46e7561c7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:40.078639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec248866-fa76-4b72-9c55-491d92d056cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:40.079786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec248866-fa76-4b72-9c55-491d92d056cc admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:40.080129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec248866-fa76-4b72-9c55-491d92d056cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:40.080361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec248866-fa76-4b72-9c55-491d92d056cc admin admin] PATH_INFO: `/v3/projects/c42afd51dd0d44aba511a26dccae48ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:40.085191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec248866-fa76-4b72-9c55-491d92d056cc admin admin] RBAC: Authorizing `identity:get_project(project_id=c42afd51dd0d44aba511a26dccae48ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:40.135592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ec248866-fa76-4b72-9c55-491d92d056cc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 15:31:40.135939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec248866-fa76-4b72-9c55-491d92d056cc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:40.140639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 3/5] 10.222.0.98 () {56 vars in 1207 bytes} [Fri Jul 15 15:31:40 2022] GET /identity/v3/projects/c42afd51dd0d44aba511a26dccae48ce => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:42.148230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e87dc8-6f9c-4fe0-82e0-1971a393bc7b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:42.148767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e87dc8-6f9c-4fe0-82e0-1971a393bc7b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:42.148767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e87dc8-6f9c-4fe0-82e0-1971a393bc7b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:42.153013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 3/6] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:42.158401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a3a5d-e82e-47b6-b02c-6e7089f64aef None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:42.158769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a3a5d-e82e-47b6-b02c-6e7089f64aef None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:42.158979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a3a5d-e82e-47b6-b02c-6e7089f64aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:42.178139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-603a3a5d-e82e-47b6-b02c-6e7089f64aef None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:42.179722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-603a3a5d-e82e-47b6-b02c-6e7089f64aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:42.187750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 4/7] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 15:31:42.210059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e87dc8-6f9c-4fe0-82e0-1971a393bc7b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:42.222160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e87dc8-6f9c-4fe0-82e0-1971a393bc7b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:42.226408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed368ac-1aeb-4d47-9ce2-0f4d6bb1dcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:42.227043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed368ac-1aeb-4d47-9ce2-0f4d6bb1dcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:42.227358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed368ac-1aeb-4d47-9ce2-0f4d6bb1dcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:42.227645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed368ac-1aeb-4d47-9ce2-0f4d6bb1dcc6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:42.235268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed368ac-1aeb-4d47-9ce2-0f4d6bb1dcc6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:42.237082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed368ac-1aeb-4d47-9ce2-0f4d6bb1dcc6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:42.246318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 4/8] 10.222.0.98 () {58 vars in 1324 bytes} [Fri Jul 15 15:31:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:31:42.251487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a3a5d-e82e-47b6-b02c-6e7089f64aef None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:42.260878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a3a5d-e82e-47b6-b02c-6e7089f64aef None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:42.265221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996ef85-0ed8-483b-9952-e8a4fb362038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:42.266070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996ef85-0ed8-483b-9952-e8a4fb362038 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:42.266358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996ef85-0ed8-483b-9952-e8a4fb362038 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:42.266636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996ef85-0ed8-483b-9952-e8a4fb362038 admin admin] PATH_INFO: `/v3/users/fb07001f45bb40c8b8c84b7d67d1c304` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:42.268038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996ef85-0ed8-483b-9952-e8a4fb362038 admin admin] RBAC: Authorizing `identity:get_user(user_id=fb07001f45bb40c8b8c84b7d67d1c304)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:42.269881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996ef85-0ed8-483b-9952-e8a4fb362038 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:42.271464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 5/9] 10.222.0.98 () {56 vars in 1198 bytes} [Fri Jul 15 15:31:42 2022] GET /identity/v3/users/fb07001f45bb40c8b8c84b7d67d1c304 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:44.351181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fda009-555a-4252-8deb-11f03956eef5 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:44.351769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fda009-555a-4252-8deb-11f03956eef5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:44.351769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fda009-555a-4252-8deb-11f03956eef5 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:44.353141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 5/10] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:44.358153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db4670-6fb5-4ef0-8e60-38136a0dbdc7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:44.358557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db4670-6fb5-4ef0-8e60-38136a0dbdc7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:44.358848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db4670-6fb5-4ef0-8e60-38136a0dbdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:44.375864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d8db4670-6fb5-4ef0-8e60-38136a0dbdc7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:44.377056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d8db4670-6fb5-4ef0-8e60-38136a0dbdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:44.400369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58a3f-1f8c-4ac8-af64-deb26a052699 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:44.400644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58a3f-1f8c-4ac8-af64-deb26a052699 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:44.400924 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d58a3f-1f8c-4ac8-af64-deb26a052699 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:44.402464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 6/11] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:31:44.407570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:44.407669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:44.407879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:44.448283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG passlib.handlers.bcrypt [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87025) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 15:31:44.448704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG passlib.handlers.bcrypt [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87025) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 15:31:44.459789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 6/12] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:31:44.482345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db4670-6fb5-4ef0-8e60-38136a0dbdc7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:44.494088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:44.495728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:44.505344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:44.585642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db4670-6fb5-4ef0-8e60-38136a0dbdc7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:44.593817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 7/13] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:31:44.616754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:44.619586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff2b0-20ab-432d-89df-59b826a8ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:44.620398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff2b0-20ab-432d-89df-59b826a8ca6c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:44.620696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff2b0-20ab-432d-89df-59b826a8ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:44.620976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aff2b0-20ab-432d-89df-59b826a8ca6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:44.633154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aff2b0-20ab-432d-89df-59b826a8ca6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-62aff2b0-20ab-432d-89df-59b826a8ca6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:31:44.644678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:44.645658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:31:44.646560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 7/14] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:31:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:31:44.651884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff2b0-20ab-432d-89df-59b826a8ca6c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:44.718625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddb890-6184-4ccc-b3c2-07cd41bf07e0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:44.720889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:44.736018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:44.753833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aff2b0-20ab-432d-89df-59b826a8ca6c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:44.756651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fab62-48c8-4b45-88df-4be1b29a0240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:44.757547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fab62-48c8-4b45-88df-4be1b29a0240 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:44.757837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fab62-48c8-4b45-88df-4be1b29a0240 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:44.758118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fab62-48c8-4b45-88df-4be1b29a0240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:44.787869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801627e-5c46-4943-b384-57f94758024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:44.788544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801627e-5c46-4943-b384-57f94758024b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:44.788839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801627e-5c46-4943-b384-57f94758024b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:44.789121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801627e-5c46-4943-b384-57f94758024b None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:44.789878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801627e-5c46-4943-b384-57f94758024b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:44.790194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801627e-5c46-4943-b384-57f94758024b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:44.791905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801627e-5c46-4943-b384-57f94758024b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:44.804952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 8/15] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:31:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:31:44.809477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801627e-5c46-4943-b384-57f94758024b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:44.826721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fab62-48c8-4b45-88df-4be1b29a0240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:44.828312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fab62-48c8-4b45-88df-4be1b29a0240 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:44.908341 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801627e-5c46-4943-b384-57f94758024b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:44.924542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 8/16] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:31:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 15:31:44.930725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fab62-48c8-4b45-88df-4be1b29a0240 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:44.933838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:44.942435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73b9fa-da6a-49ce-bd4d-4a05dc806c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:44.943891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73b9fa-da6a-49ce-bd4d-4a05dc806c04 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:44.944179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73b9fa-da6a-49ce-bd4d-4a05dc806c04 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:44.944462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73b9fa-da6a-49ce-bd4d-4a05dc806c04 None admin] PATH_INFO: `/v3/users/fb07001f45bb40c8b8c84b7d67d1c304` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:44.945683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73b9fa-da6a-49ce-bd4d-4a05dc806c04 None admin] RBAC: Authorizing `identity:get_user(user_id=fb07001f45bb40c8b8c84b7d67d1c304)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:44.947164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73b9fa-da6a-49ce-bd4d-4a05dc806c04 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:44.948820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 9/17] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:31:44 2022] GET /identity/v3/users/fb07001f45bb40c8b8c84b7d67d1c304 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:31:44.954372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73b9fa-da6a-49ce-bd4d-4a05dc806c04 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:44.968935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:45.031285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fab62-48c8-4b45-88df-4be1b29a0240 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:45.056769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73b9fa-da6a-49ce-bd4d-4a05dc806c04 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:45.065623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:45.066419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:45.066708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:45.066987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:45.089897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:45.091638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:45.096204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89830b-e776-4b6b-8db9-f693e12eefcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:45.097007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89830b-e776-4b6b-8db9-f693e12eefcf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:45.097298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89830b-e776-4b6b-8db9-f693e12eefcf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:45.097583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89830b-e776-4b6b-8db9-f693e12eefcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:45.098774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89830b-e776-4b6b-8db9-f693e12eefcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:45.100293 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89830b-e776-4b6b-8db9-f693e12eefcf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:45.101905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 10/18] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:31:45.107858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89830b-e776-4b6b-8db9-f693e12eefcf None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:31:45.112274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:45.113208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:45.113995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:31:45.113995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:31:45.113995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:31:45.113995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:31:45.113995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 9/19] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:31:45.119615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:45.175251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:45.187701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:45.206039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89830b-e776-4b6b-8db9-f693e12eefcf None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:45.219140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d7c1-c1ff-40d0-ba34-580a60c69e20 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:45.238984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be0a01-f4ca-43c7-9413-31c83647a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:45.239671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be0a01-f4ca-43c7-9413-31c83647a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:45.240469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be0a01-f4ca-43c7-9413-31c83647a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:45.240748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be0a01-f4ca-43c7-9413-31c83647a1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:45.241580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be0a01-f4ca-43c7-9413-31c83647a1a6 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.domain.id=default, user.id=fb07001f45bb40c8b8c84b7d67d1c304) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:45.241888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be0a01-f4ca-43c7-9413-31c83647a1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:45.243424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be0a01-f4ca-43c7-9413-31c83647a1a6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:45.250371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 11/20] 10.222.0.98 () {56 vars in 1322 bytes} [Fri Jul 15 15:31:45 2022] GET /identity/v3/role_assignments?user.id=fb07001f45bb40c8b8c84b7d67d1c304&scope.domain.id=default&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:31:45.252660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cab722-8ebf-40c4-941c-b4f673c91387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:45.253357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cab722-8ebf-40c4-941c-b4f673c91387 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:45.253649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cab722-8ebf-40c4-941c-b4f673c91387 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:45.253930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cab722-8ebf-40c4-941c-b4f673c91387 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:45.254667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cab722-8ebf-40c4-941c-b4f673c91387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:45.254980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cab722-8ebf-40c4-941c-b4f673c91387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:45.257055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cab722-8ebf-40c4-941c-b4f673c91387 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:45.277510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 10/21] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:31:47.401845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7f33d-b02f-4fdd-a062-92c7ad4f10e4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:47.406833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7f33d-b02f-4fdd-a062-92c7ad4f10e4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:47.407159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7f33d-b02f-4fdd-a062-92c7ad4f10e4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:47.408632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 12/22] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:47.423527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25b71-70f7-4d62-bdfe-c8c40b9a9c6a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:47.424002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25b71-70f7-4d62-bdfe-c8c40b9a9c6a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:47.424185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25b71-70f7-4d62-bdfe-c8c40b9a9c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:47.448305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-25f25b71-70f7-4d62-bdfe-c8c40b9a9c6a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:47.449536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-25f25b71-70f7-4d62-bdfe-c8c40b9a9c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:47.535343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 11/23] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:47.569614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7f33d-b02f-4fdd-a062-92c7ad4f10e4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:47.671366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce583-853b-4f33-8207-c84b6047e05a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:47.671996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce583-853b-4f33-8207-c84b6047e05a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:47.672475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ce583-853b-4f33-8207-c84b6047e05a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:47.674084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 12/24] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:31:47.680988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767f139-bc87-467e-9352-6a57f79b2b8a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:47.683027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767f139-bc87-467e-9352-6a57f79b2b8a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:47.687614 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767f139-bc87-467e-9352-6a57f79b2b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:47.712017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7f33d-b02f-4fdd-a062-92c7ad4f10e4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:47.716312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e767f139-bc87-467e-9352-6a57f79b2b8a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:47.717768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e767f139-bc87-467e-9352-6a57f79b2b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:47.762388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ac366-a693-424a-8a9e-bb8a54dcd3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:47.763194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ac366-a693-424a-8a9e-bb8a54dcd3a9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:47.763507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ac366-a693-424a-8a9e-bb8a54dcd3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:47.763780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ac366-a693-424a-8a9e-bb8a54dcd3a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:47.787661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:47.788770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ac366-a693-424a-8a9e-bb8a54dcd3a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-7b8ac366-a693-424a-8a9e-bb8a54dcd3a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:31:47.804813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:47.812417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:31:47.813153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 13/25] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:31:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:31:47.822729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ac366-a693-424a-8a9e-bb8a54dcd3a9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:47.832002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 13/26] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:31:47.835448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:47.883315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767f139-bc87-467e-9352-6a57f79b2b8a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:47.988435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ac366-a693-424a-8a9e-bb8a54dcd3a9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:48.061475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:48.062447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.062955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.063398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.064301 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:48.064766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:48.066315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:48.082618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 14/27] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:31:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:48.088916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:48.129436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767f139-bc87-467e-9352-6a57f79b2b8a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:48.214560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:48.241951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489471d7-7176-492c-9d30-36f8a354387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:48.242987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489471d7-7176-492c-9d30-36f8a354387e None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.243512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489471d7-7176-492c-9d30-36f8a354387e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.243990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489471d7-7176-492c-9d30-36f8a354387e None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.244963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489471d7-7176-492c-9d30-36f8a354387e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:48.246567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489471d7-7176-492c-9d30-36f8a354387e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:48.271829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cfe6e-23d1-4c21-b6e6-2a1236350014 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:48.309212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab187cb-e8d4-4352-8c52-4622b8152b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:48.310103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab187cb-e8d4-4352-8c52-4622b8152b76 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.310490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab187cb-e8d4-4352-8c52-4622b8152b76 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.310775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab187cb-e8d4-4352-8c52-4622b8152b76 None admin] PATH_INFO: `/v3/users/fb07001f45bb40c8b8c84b7d67d1c304` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.312019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab187cb-e8d4-4352-8c52-4622b8152b76 None admin] RBAC: Authorizing `identity:get_user(user_id=fb07001f45bb40c8b8c84b7d67d1c304)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:48.313547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab187cb-e8d4-4352-8c52-4622b8152b76 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:48.318757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 15/28] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:31:48 2022] GET /identity/v3/users/fb07001f45bb40c8b8c84b7d67d1c304 => generated 247 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:48.318757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7da7-2339-4f30-b1db-6806bdf5d75f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.318757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7da7-2339-4f30-b1db-6806bdf5d75f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.318757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7da7-2339-4f30-b1db-6806bdf5d75f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.318757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 16/29] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:48.320824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc050-bc3e-4a10-a81d-0dec3487bf53 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.320824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc050-bc3e-4a10-a81d-0dec3487bf53 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.320824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc050-bc3e-4a10-a81d-0dec3487bf53 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.322193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 17/30] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:48.322745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fc050-bc3e-4a10-a81d-0dec3487bf53 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:48.437918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fc050-bc3e-4a10-a81d-0dec3487bf53 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:48.473300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad55da-3ec0-45ec-9b75-d7f14382aa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:48.474151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad55da-3ec0-45ec-9b75-d7f14382aa95 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.475306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad55da-3ec0-45ec-9b75-d7f14382aa95 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.475681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad55da-3ec0-45ec-9b75-d7f14382aa95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.476887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad55da-3ec0-45ec-9b75-d7f14382aa95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:48.478481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad55da-3ec0-45ec-9b75-d7f14382aa95 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:48.484943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 18/31] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:48.484943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e2b76-4ce8-4762-9ffd-b403a6a8d867 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.484943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e2b76-4ce8-4762-9ffd-b403a6a8d867 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.484943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e2b76-4ce8-4762-9ffd-b403a6a8d867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.489266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:48.502333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ea8e2b76-4ce8-4762-9ffd-b403a6a8d867 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:48.503500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ea8e2b76-4ce8-4762-9ffd-b403a6a8d867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:48.511769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 14/32] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Jul 15 15:31:47 2022] POST /identity/v3/roles => generated 219 bytes in 629 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:31:48.515128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdbb93-9232-4c25-9e68-9263d493a1d4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.515656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdbb93-9232-4c25-9e68-9263d493a1d4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.516499 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdbb93-9232-4c25-9e68-9263d493a1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.561409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-96bdbb93-9232-4c25-9e68-9263d493a1d4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:48.563006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-96bdbb93-9232-4c25-9e68-9263d493a1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:48.629050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:48.637706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 19/33] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:48.638953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e2b76-4ce8-4762-9ffd-b403a6a8d867 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:48.705160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 15/34] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:48.705887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdbb93-9232-4c25-9e68-9263d493a1d4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:48.752021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e2b76-4ce8-4762-9ffd-b403a6a8d867 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:48.783530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:48.788921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae13d09-3cd6-4222-95ba-a5a6b7aaa4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:48.789811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13d09-3cd6-4222-95ba-a5a6b7aaa4a5 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.795779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13d09-3cd6-4222-95ba-a5a6b7aaa4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.795779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13d09-3cd6-4222-95ba-a5a6b7aaa4a5 None admin] PATH_INFO: `/v3/domains/default/users/fb07001f45bb40c8b8c84b7d67d1c304/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.795779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae13d09-3cd6-4222-95ba-a5a6b7aaa4a5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fb07001f45bb40c8b8c84b7d67d1c304, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:48.796435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae13d09-3cd6-4222-95ba-a5a6b7aaa4a5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:48.840408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdbb93-9232-4c25-9e68-9263d493a1d4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:48.881383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:48.882262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:48.882571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:48.882846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:48.894440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:48.896727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:31:48.916181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:48.917007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:48.917781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:31:48.917781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:31:48.917781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:31:48.917781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:31:48.917781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 16/35] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:31:48.918558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:48.940301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:49.028819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405026c6-9988-43c8-90ca-4a891c188cad None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:49.029577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 20/36] 10.222.0.98 () {58 vars in 1361 bytes} [Fri Jul 15 15:31:48 2022] PUT /identity/v3/domains/default/users/fb07001f45bb40c8b8c84b7d67d1c304/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:31:49.030816 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae13d09-3cd6-4222-95ba-a5a6b7aaa4a5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:49.073303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94f53e-2f85-4993-9ede-f43319fd5660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:49.074398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f53e-2f85-4993-9ede-f43319fd5660 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.074900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f53e-2f85-4993-9ede-f43319fd5660 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.075430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f53e-2f85-4993-9ede-f43319fd5660 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.076790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94f53e-2f85-4993-9ede-f43319fd5660 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:49.078741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94f53e-2f85-4993-9ede-f43319fd5660 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:49.155991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 17/37] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Jul 15 15:31:48 2022] POST /identity/v3/roles => generated 225 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:31:49.200748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae13d09-3cd6-4222-95ba-a5a6b7aaa4a5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:49.236931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2f54d-fca7-425e-b4e4-099be83d5aa4 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.237351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2f54d-fca7-425e-b4e4-099be83d5aa4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.237539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2f54d-fca7-425e-b4e4-099be83d5aa4 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.238891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 18/38] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:31:49.242159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:49.243151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.243667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.244163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.244723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eac26-1817-4bed-9718-9172c04b6b58 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.245020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eac26-1817-4bed-9718-9172c04b6b58 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.245307 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eac26-1817-4bed-9718-9172c04b6b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.245850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:49.246376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:49.248569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:49.266762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-656eac26-1817-4bed-9718-9172c04b6b58 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:49.268059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-656eac26-1817-4bed-9718-9172c04b6b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:49.275014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 21/39] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:49.281073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:49.350821 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:49.369878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 19/40] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:49.371428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f31a0a-0b49-4632-8393-89d5560a1d66 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.371937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f31a0a-0b49-4632-8393-89d5560a1d66 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.372227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f31a0a-0b49-4632-8393-89d5560a1d66 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.373542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 20/41] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:49.382708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c14869-72d9-4861-af52-e99cc33be6f3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.382708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c14869-72d9-4861-af52-e99cc33be6f3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.382708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c14869-72d9-4861-af52-e99cc33be6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.403184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-83c14869-72d9-4861-af52-e99cc33be6f3 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:49.404567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-83c14869-72d9-4861-af52-e99cc33be6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:49.411320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:49.422358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41894aee-2023-4226-80c1-8dc76181f5d6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:49.469896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18a69-97d4-438e-a22f-a515640812ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:49.470635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18a69-97d4-438e-a22f-a515640812ea None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.470934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18a69-97d4-438e-a22f-a515640812ea None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.471220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad18a69-97d4-438e-a22f-a515640812ea None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.471970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad18a69-97d4-438e-a22f-a515640812ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:49.473450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad18a69-97d4-438e-a22f-a515640812ea None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:49.499316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 21/42] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:49.499316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c14869-72d9-4861-af52-e99cc33be6f3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:49.590564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c14869-72d9-4861-af52-e99cc33be6f3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:49.605709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 22/43] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:31:49 2022] POST /identity/v3/projects => generated 301 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:31:49.616463 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18a69-97d4-438e-a22f-a515640812ea None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:49.628159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a2600-1a46-429b-980b-06b9ef2a57e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:49.628860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a2600-1a46-429b-980b-06b9ef2a57e9 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.629151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a2600-1a46-429b-980b-06b9ef2a57e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.629432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a2600-1a46-429b-980b-06b9ef2a57e9 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.630262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a2600-1a46-429b-980b-06b9ef2a57e9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:49.632714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a2600-1a46-429b-980b-06b9ef2a57e9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:49.720627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:49.728298 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 22/44] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Jul 15 15:31:49 2022] POST /identity/v3/roles => generated 223 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:31:49.859786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad18a69-97d4-438e-a22f-a515640812ea None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:49.863959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:49.901147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee94314-5540-4b6a-a45f-fd663879fc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:49.901976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee94314-5540-4b6a-a45f-fd663879fc03 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:49.902879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee94314-5540-4b6a-a45f-fd663879fc03 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:49.903573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee94314-5540-4b6a-a45f-fd663879fc03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:49.905375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee94314-5540-4b6a-a45f-fd663879fc03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:49.907096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee94314-5540-4b6a-a45f-fd663879fc03 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:49.908769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 23/45] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:49.914335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a2600-1a46-429b-980b-06b9ef2a57e9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:50.017987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a2600-1a46-429b-980b-06b9ef2a57e9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:50.051384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb2337-3e85-4588-993e-1a36ec8f2784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:50.052048 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb2337-3e85-4588-993e-1a36ec8f2784 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:50.052355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb2337-3e85-4588-993e-1a36ec8f2784 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:50.053163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb2337-3e85-4588-993e-1a36ec8f2784 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:50.054126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb2337-3e85-4588-993e-1a36ec8f2784 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:50.055601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb2337-3e85-4588-993e-1a36ec8f2784 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:50.148785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 23/46] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:31:49 2022] POST /identity/v3/projects => generated 312 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:31:51.215633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10448d22-bbc8-4f09-a619-bf1cbce9c423 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:51.216221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10448d22-bbc8-4f09-a619-bf1cbce9c423 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:51.216221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10448d22-bbc8-4f09-a619-bf1cbce9c423 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:51.217622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 24/47] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:51.222828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6c402-f1cf-46a8-acb5-19c5b18f8bb7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:51.223241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6c402-f1cf-46a8-acb5-19c5b18f8bb7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:51.223427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6c402-f1cf-46a8-acb5-19c5b18f8bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:51.240809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-67c6c402-f1cf-46a8-acb5-19c5b18f8bb7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:51.242047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-67c6c402-f1cf-46a8-acb5-19c5b18f8bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:51.289962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:51.309489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 24/48] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:51.332905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10448d22-bbc8-4f09-a619-bf1cbce9c423 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:51.436183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10448d22-bbc8-4f09-a619-bf1cbce9c423 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:51.470790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e72c7e-2032-4875-b5a0-e9528b1bad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:51.471551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e72c7e-2032-4875-b5a0-e9528b1bad45 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:51.471842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e72c7e-2032-4875-b5a0-e9528b1bad45 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:51.472122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e72c7e-2032-4875-b5a0-e9528b1bad45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:51.482642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e72c7e-2032-4875-b5a0-e9528b1bad45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-97e72c7e-2032-4875-b5a0-e9528b1bad45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:31:51.490718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:51.491602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:31:51.492335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 25/49] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:31:51.499548 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6c402-f1cf-46a8-acb5-19c5b18f8bb7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:51.595995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6c402-f1cf-46a8-acb5-19c5b18f8bb7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:51.629690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:51.630359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:51.631171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:51.631489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:51.632247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:51.632562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:51.633974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:51.645454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 25/50] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:31:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:51.651851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e72c7e-2032-4875-b5a0-e9528b1bad45 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:51.683214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:51.753296 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e72c7e-2032-4875-b5a0-e9528b1bad45 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:51.788709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4d1da-9cc7-40a1-b3ec-b6fdb29bba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:51.789588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4d1da-9cc7-40a1-b3ec-b6fdb29bba3d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:51.789883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4d1da-9cc7-40a1-b3ec-b6fdb29bba3d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:51.790170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4d1da-9cc7-40a1-b3ec-b6fdb29bba3d None admin] PATH_INFO: `/v3/users/fb07001f45bb40c8b8c84b7d67d1c304` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:51.791411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4d1da-9cc7-40a1-b3ec-b6fdb29bba3d None admin] RBAC: Authorizing `identity:get_user(user_id=fb07001f45bb40c8b8c84b7d67d1c304)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:51.792892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4d1da-9cc7-40a1-b3ec-b6fdb29bba3d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:51.794868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 26/51] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:31:51 2022] GET /identity/v3/users/fb07001f45bb40c8b8c84b7d67d1c304 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:51.800260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:51.843492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:51.902814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f206ad-a923-48be-865c-1ae6eb621702 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:51.936696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed344fad-3c17-4cb3-bce2-885b0167b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:51.937497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed344fad-3c17-4cb3-bce2-885b0167b07a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:51.937788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed344fad-3c17-4cb3-bce2-885b0167b07a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:51.938071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed344fad-3c17-4cb3-bce2-885b0167b07a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:51.939260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed344fad-3c17-4cb3-bce2-885b0167b07a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:51.940763 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed344fad-3c17-4cb3-bce2-885b0167b07a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:51.942398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 26/52] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:51.948946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4d1da-9cc7-40a1-b3ec-b6fdb29bba3d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:52.043804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:52.046745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4d1da-9cc7-40a1-b3ec-b6fdb29bba3d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:52.080715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea01f32-1b2c-4d28-a1df-d839e0e264c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:52.081379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea01f32-1b2c-4d28-a1df-d839e0e264c7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:52.081672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea01f32-1b2c-4d28-a1df-d839e0e264c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:52.081956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea01f32-1b2c-4d28-a1df-d839e0e264c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:52.082795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea01f32-1b2c-4d28-a1df-d839e0e264c7 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.domain.id=default, user.id=fb07001f45bb40c8b8c84b7d67d1c304) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:52.083114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea01f32-1b2c-4d28-a1df-d839e0e264c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:52.084701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea01f32-1b2c-4d28-a1df-d839e0e264c7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:52.090695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 27/53] 10.222.0.98 () {56 vars in 1322 bytes} [Fri Jul 15 15:31:51 2022] GET /identity/v3/role_assignments?user.id=fb07001f45bb40c8b8c84b7d67d1c304&scope.domain.id=default&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:52.148871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c8ff0-31bd-42c5-ba76-e2d069bee148 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:52.149160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c8ff0-31bd-42c5-ba76-e2d069bee148 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:52.149446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c8ff0-31bd-42c5-ba76-e2d069bee148 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:52.150846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 27/54] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:52.156656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91f8e-b3b8-4c23-a8c8-724f09227c7b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:52.156955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91f8e-b3b8-4c23-a8c8-724f09227c7b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:52.157242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b91f8e-b3b8-4c23-a8c8-724f09227c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:52.175437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-47b91f8e-b3b8-4c23-a8c8-724f09227c7b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:52.177004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-47b91f8e-b3b8-4c23-a8c8-724f09227c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:52.249937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 28/55] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:52.273186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c8ff0-31bd-42c5-ba76-e2d069bee148 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:52.477133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c8ff0-31bd-42c5-ba76-e2d069bee148 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:52.487314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:52.510132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7207990-0be2-44b7-9cc2-0e56a55e7626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:52.510918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7207990-0be2-44b7-9cc2-0e56a55e7626 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:52.511208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7207990-0be2-44b7-9cc2-0e56a55e7626 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:52.511518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7207990-0be2-44b7-9cc2-0e56a55e7626 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:52.512703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7207990-0be2-44b7-9cc2-0e56a55e7626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:52.514475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7207990-0be2-44b7-9cc2-0e56a55e7626 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:52.516374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 28/56] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:52.521863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91f8e-b3b8-4c23-a8c8-724f09227c7b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:52.612192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:52.619076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b91f8e-b3b8-4c23-a8c8-724f09227c7b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:52.652082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3462c-27d9-47dd-a2ca-f90d274b8399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:52.652743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3462c-27d9-47dd-a2ca-f90d274b8399 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:52.653034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3462c-27d9-47dd-a2ca-f90d274b8399 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:52.653314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3462c-27d9-47dd-a2ca-f90d274b8399 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:52.654014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3462c-27d9-47dd-a2ca-f90d274b8399 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:52.655516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3462c-27d9-47dd-a2ca-f90d274b8399 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:52.729552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 29/57] 10.222.0.98 () {60 vars in 1140 bytes} [Fri Jul 15 15:31:52 2022] POST /identity/v3/projects => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:31:54.720261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21a85-12ab-476e-a41b-1deee72f2e96 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:54.720747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21a85-12ab-476e-a41b-1deee72f2e96 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:54.720849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21a85-12ab-476e-a41b-1deee72f2e96 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:54.722199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 29/58] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:54.727951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb294ff-c096-430e-8c12-ac10e2a963ab None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:54.728310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb294ff-c096-430e-8c12-ac10e2a963ab None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:54.728529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb294ff-c096-430e-8c12-ac10e2a963ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:54.744999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2fb294ff-c096-430e-8c12-ac10e2a963ab None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:54.746200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2fb294ff-c096-430e-8c12-ac10e2a963ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:54.810531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 30/59] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:54.834454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21a85-12ab-476e-a41b-1deee72f2e96 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:54.932951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21a85-12ab-476e-a41b-1deee72f2e96 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:54.966304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d97301-baaf-496b-afa6-eae26fb250d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:54.967088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d97301-baaf-496b-afa6-eae26fb250d1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:54.967404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d97301-baaf-496b-afa6-eae26fb250d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:54.967684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d97301-baaf-496b-afa6-eae26fb250d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:54.968849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d97301-baaf-496b-afa6-eae26fb250d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:54.970409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d97301-baaf-496b-afa6-eae26fb250d1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:54.971995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 30/60] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:54.977725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb294ff-c096-430e-8c12-ac10e2a963ab None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:55.052885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:55.075487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb294ff-c096-430e-8c12-ac10e2a963ab None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:55.108634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65672ab9-0848-4e21-b2ce-80ade5f54c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:55.109297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65672ab9-0848-4e21-b2ce-80ade5f54c7a None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:55.109585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65672ab9-0848-4e21-b2ce-80ade5f54c7a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:55.109864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65672ab9-0848-4e21-b2ce-80ade5f54c7a None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:55.110567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65672ab9-0848-4e21-b2ce-80ade5f54c7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:55.112269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65672ab9-0848-4e21-b2ce-80ade5f54c7a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:55.199755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 31/61] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:31:54 2022] POST /identity/v3/users => generated 275 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:31:58.263566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c316a0-cc9c-4fe7-ac00-b0e7b566c0f9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.264541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c316a0-cc9c-4fe7-ac00-b0e7b566c0f9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.264981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c316a0-cc9c-4fe7-ac00-b0e7b566c0f9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.266501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 31/62] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:58.272037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2037d9f-5608-4c26-a3f4-057c730007c0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.273040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2037d9f-5608-4c26-a3f4-057c730007c0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.273470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2037d9f-5608-4c26-a3f4-057c730007c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.300294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c2037d9f-5608-4c26-a3f4-057c730007c0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:58.301756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c2037d9f-5608-4c26-a3f4-057c730007c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:58.337427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8098-3a5b-47f4-b82c-4a52b2ae769b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.337935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8098-3a5b-47f4-b82c-4a52b2ae769b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.338355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8098-3a5b-47f4-b82c-4a52b2ae769b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.339917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 32/63] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:58.347335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053cb4b-73bc-45b6-a515-d65f42ba15df None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.347868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053cb4b-73bc-45b6-a515-d65f42ba15df None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.348346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053cb4b-73bc-45b6-a515-d65f42ba15df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.368631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f053cb4b-73bc-45b6-a515-d65f42ba15df None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:58.371873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f053cb4b-73bc-45b6-a515-d65f42ba15df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:58.411090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 32/64] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:58.434999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2037d9f-5608-4c26-a3f4-057c730007c0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:58.453186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:58.512072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 33/65] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 15 15:31:58.535414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:58.546093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053cb4b-73bc-45b6-a515-d65f42ba15df None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:58.600250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2037d9f-5608-4c26-a3f4-057c730007c0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:58.656038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01aaa18-8810-4c84-a8d6-9b105b3e3b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:58.656855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aaa18-8810-4c84-a8d6-9b105b3e3b15 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.657146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aaa18-8810-4c84-a8d6-9b105b3e3b15 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.657427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aaa18-8810-4c84-a8d6-9b105b3e3b15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.680278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01aaa18-8810-4c84-a8d6-9b105b3e3b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a01aaa18-8810-4c84-a8d6-9b105b3e3b15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:31:58.688167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:58.689182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:31:58.690123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:31:58.692173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 33/66] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:31:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:31:58.693593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac80d-ba4f-4e4e-b109-f3a4d6bae833 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.694049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac80d-ba4f-4e4e-b109-f3a4d6bae833 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.694494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac80d-ba4f-4e4e-b109-f3a4d6bae833 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.695969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 34/67] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:58.700855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ac80d-ba4f-4e4e-b109-f3a4d6bae833 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:58.745506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053cb4b-73bc-45b6-a515-d65f42ba15df None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:58.789553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b340c-d4b8-483f-839c-f4bf83aa661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:58.790628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b340c-d4b8-483f-839c-f4bf83aa661a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.791063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b340c-d4b8-483f-839c-f4bf83aa661a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.791518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b340c-d4b8-483f-839c-f4bf83aa661a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.806182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b340c-d4b8-483f-839c-f4bf83aa661a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c86b340c-d4b8-483f-839c-f4bf83aa661a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:31:58.819114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:58.820019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:31:58.820768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 34/68] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:31:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:31:58.822466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa6c76-7dee-4838-b2a7-c0a62d965e45 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.822779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa6c76-7dee-4838-b2a7-c0a62d965e45 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.823075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa6c76-7dee-4838-b2a7-c0a62d965e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.823745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ac80d-ba4f-4e4e-b109-f3a4d6bae833 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:58.840268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3efa6c76-7dee-4838-b2a7-c0a62d965e45 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:58.841567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3efa6c76-7dee-4838-b2a7-c0a62d965e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:58.850002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:58.861699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:58.862652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.863154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.866888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.866888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:58.866888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:58.866888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:58.872315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:58.876365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 35/69] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:58.877134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:58.917290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 35/70] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:58.919093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35afcb-5b16-44b1-b46e-d8220079ea20 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:58.919595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35afcb-5b16-44b1-b46e-d8220079ea20 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:58.920050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35afcb-5b16-44b1-b46e-d8220079ea20 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:58.921698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 36/71] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:58.922562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35afcb-5b16-44b1-b46e-d8220079ea20 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:58.995309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a4e7c-a657-4696-89bc-2e5e96cc2e86 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.030394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab2ddf-f036-42fe-98cd-c1ba07aa6cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.031081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2ddf-f036-42fe-98cd-c1ba07aa6cc7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.031413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2ddf-f036-42fe-98cd-c1ba07aa6cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.031701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2ddf-f036-42fe-98cd-c1ba07aa6cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.032454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab2ddf-f036-42fe-98cd-c1ba07aa6cc7 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:59.032845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab2ddf-f036-42fe-98cd-c1ba07aa6cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:59.034476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab2ddf-f036-42fe-98cd-c1ba07aa6cc7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:59.052262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35afcb-5b16-44b1-b46e-d8220079ea20 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.053009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 36/72] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:31:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:59.053009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712c652-d584-469a-a426-d086962a7978 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.053009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712c652-d584-469a-a426-d086962a7978 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.053009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712c652-d584-469a-a426-d086962a7978 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.053009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 37/73] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:31:59.053009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f3297-8265-4940-9023-89c0cd9f986b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.053009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f3297-8265-4940-9023-89c0cd9f986b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.053451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f3297-8265-4940-9023-89c0cd9f986b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.065514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8a1f3297-8265-4940-9023-89c0cd9f986b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:59.066733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8a1f3297-8265-4940-9023-89c0cd9f986b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:59.075564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bf4b5-90ba-4183-8d29-2969db28d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.076449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bf4b5-90ba-4183-8d29-2969db28d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.076737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bf4b5-90ba-4183-8d29-2969db28d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.077019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bf4b5-90ba-4183-8d29-2969db28d2f1 None admin] PATH_INFO: `/v3/users/fb07001f45bb40c8b8c84b7d67d1c304` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.078459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bf4b5-90ba-4183-8d29-2969db28d2f1 None admin] RBAC: Authorizing `identity:get_user(user_id=fb07001f45bb40c8b8c84b7d67d1c304)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:59.079974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bf4b5-90ba-4183-8d29-2969db28d2f1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:59.081979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 37/74] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:31:58 2022] GET /identity/v3/users/fb07001f45bb40c8b8c84b7d67d1c304 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:59.082707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bf4b5-90ba-4183-8d29-2969db28d2f1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.112203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:59.117132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:59.136425 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 38/75] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:59.137349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f3297-8265-4940-9023-89c0cd9f986b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.183547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bf4b5-90ba-4183-8d29-2969db28d2f1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.216623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611285c-3fe9-445e-b570-dc491a3b37f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.217430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611285c-3fe9-445e-b570-dc491a3b37f1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.217729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611285c-3fe9-445e-b570-dc491a3b37f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.218012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611285c-3fe9-445e-b570-dc491a3b37f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.228039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611285c-3fe9-445e-b570-dc491a3b37f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7611285c-3fe9-445e-b570-dc491a3b37f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:31:59.236237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:59.237299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:31:59.238240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 38/76] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:31:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:31:59.238884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65bfb-baad-44df-9f45-dbc4f0e5b30e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.238884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65bfb-baad-44df-9f45-dbc4f0e5b30e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.239129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65bfb-baad-44df-9f45-dbc4f0e5b30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.244413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f3297-8265-4940-9023-89c0cd9f986b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.256425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-72e65bfb-baad-44df-9f45-dbc4f0e5b30e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:59.257848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-72e65bfb-baad-44df-9f45-dbc4f0e5b30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:59.278666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862c021-622c-4b13-a654-e256baa9e874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.279603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862c021-622c-4b13-a654-e256baa9e874 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.279900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862c021-622c-4b13-a654-e256baa9e874 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.280188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862c021-622c-4b13-a654-e256baa9e874 None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.294793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862c021-622c-4b13-a654-e256baa9e874 None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:59.296374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862c021-622c-4b13-a654-e256baa9e874 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:59.298171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 39/77] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:31:59 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:59.298965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862c021-622c-4b13-a654-e256baa9e874 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.336299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 39/78] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:31:59.337477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:59.337988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e65bfb-baad-44df-9f45-dbc4f0e5b30e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.367185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:59.406486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862c021-622c-4b13-a654-e256baa9e874 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.440339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc804658-0b22-4869-9814-d113e7281ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.440794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e65bfb-baad-44df-9f45-dbc4f0e5b30e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.441000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc804658-0b22-4869-9814-d113e7281ef7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.441293 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc804658-0b22-4869-9814-d113e7281ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.441576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc804658-0b22-4869-9814-d113e7281ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.474600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de29bb-585f-4c75-a6de-8e128e7de2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.475426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de29bb-585f-4c75-a6de-8e128e7de2ac None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.475720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de29bb-585f-4c75-a6de-8e128e7de2ac None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.476021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de29bb-585f-4c75-a6de-8e128e7de2ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.477187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de29bb-585f-4c75-a6de-8e128e7de2ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:59.478645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de29bb-585f-4c75-a6de-8e128e7de2ac None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:59.480464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 40/79] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:31:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:59.481056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de29bb-585f-4c75-a6de-8e128e7de2ac None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.511772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc804658-0b22-4869-9814-d113e7281ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:59.514226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc804658-0b22-4869-9814-d113e7281ef7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:59.547283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:59.572552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:59.581615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de29bb-585f-4c75-a6de-8e128e7de2ac None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.613794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 40/80] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:31:59.614567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc804658-0b22-4869-9814-d113e7281ef7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.620906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.621589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.621878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.622158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.622918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:31:59.623226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:59.624672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:59.633556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 41/81] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:31:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:31:59.634363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.716387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc804658-0b22-4869-9814-d113e7281ef7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.734758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e468f-6396-46ad-9a3b-62a17f5215e8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.749220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2306a76-8285-4628-b6e4-d155d26bd935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.749873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2306a76-8285-4628-b6e4-d155d26bd935 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.750167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2306a76-8285-4628-b6e4-d155d26bd935 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.750455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2306a76-8285-4628-b6e4-d155d26bd935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.753102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:59.767581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946e21b-2a9a-481d-a410-9c820f871b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.768374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946e21b-2a9a-481d-a410-9c820f871b91 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.773098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946e21b-2a9a-481d-a410-9c820f871b91 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.773098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946e21b-2a9a-481d-a410-9c820f871b91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.779970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946e21b-2a9a-481d-a410-9c820f871b91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f946e21b-2a9a-481d-a410-9c820f871b91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:31:59.787523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:31:59.788408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 42/82] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:31:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:31:59.794493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946e21b-2a9a-481d-a410-9c820f871b91 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.810336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:59.819703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2306a76-8285-4628-b6e4-d155d26bd935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:59.824769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2306a76-8285-4628-b6e4-d155d26bd935 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:59.897630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946e21b-2a9a-481d-a410-9c820f871b91 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:59.923659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 41/83] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:31:59.924506 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2306a76-8285-4628-b6e4-d155d26bd935 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:59.936107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8417ac5-d13c-4255-b648-56cd84d13f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:59.936783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8417ac5-d13c-4255-b648-56cd84d13f0a None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:59.937080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8417ac5-d13c-4255-b648-56cd84d13f0a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:59.937367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8417ac5-d13c-4255-b648-56cd84d13f0a None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:59.938079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8417ac5-d13c-4255-b648-56cd84d13f0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:59.939583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8417ac5-d13c-4255-b648-56cd84d13f0a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:59.955878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:00.020121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2306a76-8285-4628-b6e4-d155d26bd935 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.020761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 43/84] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:31:59 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:00.021684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8417ac5-d13c-4255-b648-56cd84d13f0a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.066754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8929f5-61cf-465d-b351-63a6ea1ad1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.067685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8929f5-61cf-465d-b351-63a6ea1ad1be None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.067980 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8929f5-61cf-465d-b351-63a6ea1ad1be None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.069000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8929f5-61cf-465d-b351-63a6ea1ad1be None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.070427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8929f5-61cf-465d-b351-63a6ea1ad1be None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:00.071988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8929f5-61cf-465d-b351-63a6ea1ad1be None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:00.073829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 42/85] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:31:59 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:00.074657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8929f5-61cf-465d-b351-63a6ea1ad1be None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.113585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:00.147416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8417ac5-d13c-4255-b648-56cd84d13f0a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.190864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce437e6-d15d-4c00-ad52-8b57bb0cd2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.191762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce437e6-d15d-4c00-ad52-8b57bb0cd2cd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.192057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce437e6-d15d-4c00-ad52-8b57bb0cd2cd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.192341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce437e6-d15d-4c00-ad52-8b57bb0cd2cd None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.193178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8929f5-61cf-465d-b351-63a6ea1ad1be None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.193619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce437e6-d15d-4c00-ad52-8b57bb0cd2cd None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:00.194998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:00.195108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce437e6-d15d-4c00-ad52-8b57bb0cd2cd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:00.196944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 44/86] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:00.198892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce437e6-d15d-4c00-ad52-8b57bb0cd2cd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.242816 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.243591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.243917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.244187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.244945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:00.245263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:00.247382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:00.259876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 43/87] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:00.260580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.322774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce437e6-d15d-4c00-ad52-8b57bb0cd2cd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.369867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bca526-f154-4ea3-83a8-4918c3355f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.371792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca526-f154-4ea3-83a8-4918c3355f2d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.372241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca526-f154-4ea3-83a8-4918c3355f2d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.372671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bca526-f154-4ea3-83a8-4918c3355f2d None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.374563 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bca526-f154-4ea3-83a8-4918c3355f2d None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:00.376751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bca526-f154-4ea3-83a8-4918c3355f2d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:00.379894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 45/88] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:00.381489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bca526-f154-4ea3-83a8-4918c3355f2d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.390149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8204e-c11d-41fc-b38f-93df879c8f03 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.393623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:00.441454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.443601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.444385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.445282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.446523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=b7fddf3939de4bb99741de8e3307565f, user.id=fb07001f45bb40c8b8c84b7d67d1c304) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:00.447442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:00.449581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:00.457876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 44/89] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/role_assignments?user.id=fb07001f45bb40c8b8c84b7d67d1c304&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:00.460056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.497544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bca526-f154-4ea3-83a8-4918c3355f2d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.506129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:00.538120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6a036-ef6a-4744-a2c4-6c32ff4f4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.538694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6a036-ef6a-4744-a2c4-6c32ff4f4c45 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.539061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6a036-ef6a-4744-a2c4-6c32ff4f4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.539468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6a036-ef6a-4744-a2c4-6c32ff4f4c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.593289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362b967-f9fa-462a-a45b-35d44b23618b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.615636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6a036-ef6a-4744-a2c4-6c32ff4f4c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:00.617259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6a036-ef6a-4744-a2c4-6c32ff4f4c45 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:00.634614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afc847-8f40-4616-a3b7-a87e23bb14fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.635287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:00.635538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afc847-8f40-4616-a3b7-a87e23bb14fe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.635834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afc847-8f40-4616-a3b7-a87e23bb14fe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.636119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afc847-8f40-4616-a3b7-a87e23bb14fe None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.637432 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afc847-8f40-4616-a3b7-a87e23bb14fe None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:00.638915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afc847-8f40-4616-a3b7-a87e23bb14fe None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:00.640918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 45/90] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:00.641711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afc847-8f40-4616-a3b7-a87e23bb14fe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.731575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 46/91] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:00.732488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6a036-ef6a-4744-a2c4-6c32ff4f4c45 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.770101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:00.777457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afc847-8f40-4616-a3b7-a87e23bb14fe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.814444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.815266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.815713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.816108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.817304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] RBAC: Adding query filter params (role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=b7fddf3939de4bb99741de8e3307565f, user.id=c956159832a6436bbf4c4a3517bd6170) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:00.817796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:00.824927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:00.830944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 46/92] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/role_assignments?user.id=c956159832a6436bbf4c4a3517bd6170&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:00.831917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:00.844859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6a036-ef6a-4744-a2c4-6c32ff4f4c45 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:00.879746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:00.891858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0433b1e-d1d8-4aa0-a0cf-4500dfef3fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:00.892601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0433b1e-d1d8-4aa0-a0cf-4500dfef3fc4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:00.892897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0433b1e-d1d8-4aa0-a0cf-4500dfef3fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:00.893182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0433b1e-d1d8-4aa0-a0cf-4500dfef3fc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:00.967128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f10cc-6903-4591-96fb-4f7a4ce97646 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:01.003794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f31cf7-9a7c-48f2-9166-cc289f096f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:01.004621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f31cf7-9a7c-48f2-9166-cc289f096f35 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:01.005070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f31cf7-9a7c-48f2-9166-cc289f096f35 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:01.005357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f31cf7-9a7c-48f2-9166-cc289f096f35 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:01.006703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f31cf7-9a7c-48f2-9166-cc289f096f35 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:01.008231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f31cf7-9a7c-48f2-9166-cc289f096f35 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:01.011158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 47/93] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:01.016016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f31cf7-9a7c-48f2-9166-cc289f096f35 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:01.016209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0433b1e-d1d8-4aa0-a0cf-4500dfef3fc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:01.017723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0433b1e-d1d8-4aa0-a0cf-4500dfef3fc4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:01.094497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:01.180034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f31cf7-9a7c-48f2-9166-cc289f096f35 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:01.224145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:01.224936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:01.225235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:01.225677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:01.226530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] RBAC: Adding query filter params (role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=b7fddf3939de4bb99741de8e3307565f, user.id=c956159832a6436bbf4c4a3517bd6170) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:01.226849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:01.229002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:01.236361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 48/94] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:01 2022] GET /identity/v3/role_assignments?user.id=c956159832a6436bbf4c4a3517bd6170&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:01.250782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 47/95] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:01.256343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:01.387872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:01.398088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbaccf-8fa9-4502-8fd1-44f4526ab449 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:01.436011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7e951-fa46-477c-877c-513e46d7ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:01.436909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e951-fa46-477c-877c-513e46d7ac4d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:01.437202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e951-fa46-477c-877c-513e46d7ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:01.437495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7e951-fa46-477c-877c-513e46d7ac4d None admin] PATH_INFO: `/v3/projects/d0157c96b92744a4b2956b1ef2c7d110` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:01.438764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7e951-fa46-477c-877c-513e46d7ac4d None admin] RBAC: Authorizing `identity:get_project(project_id=d0157c96b92744a4b2956b1ef2c7d110)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:01.440308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7e951-fa46-477c-877c-513e46d7ac4d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:01.442065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 49/96] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:01 2022] GET /identity/v3/projects/d0157c96b92744a4b2956b1ef2c7d110 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:01.452467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0433b1e-d1d8-4aa0-a0cf-4500dfef3fc4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:01.467859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:01.577554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0433b1e-d1d8-4aa0-a0cf-4500dfef3fc4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:01.613357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca56ae-9717-4d3e-bdf7-e3bfba9c151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:01.614022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca56ae-9717-4d3e-bdf7-e3bfba9c151c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:01.614314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca56ae-9717-4d3e-bdf7-e3bfba9c151c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:01.614596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca56ae-9717-4d3e-bdf7-e3bfba9c151c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:01.615516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca56ae-9717-4d3e-bdf7-e3bfba9c151c None admin] RBAC: Adding query filter params (role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=d0157c96b92744a4b2956b1ef2c7d110, user.id=c956159832a6436bbf4c4a3517bd6170) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:01.615835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca56ae-9717-4d3e-bdf7-e3bfba9c151c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:01.617283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca56ae-9717-4d3e-bdf7-e3bfba9c151c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:01.625737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 48/97] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:01 2022] GET /identity/v3/role_assignments?user.id=c956159832a6436bbf4c4a3517bd6170&scope.project.id=d0157c96b92744a4b2956b1ef2c7d110&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:02.578874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be82b39-35b5-496a-9025-8af8257c51c8 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:02.579919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be82b39-35b5-496a-9025-8af8257c51c8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:02.580417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be82b39-35b5-496a-9025-8af8257c51c8 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:02.582022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 50/98] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:02.588069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a099-8d68-46d1-b286-98c4ea143de0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:02.588936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a099-8d68-46d1-b286-98c4ea143de0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:02.589427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87a099-8d68-46d1-b286-98c4ea143de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:02.608046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cd87a099-8d68-46d1-b286-98c4ea143de0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:02.609604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd87a099-8d68-46d1-b286-98c4ea143de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:02.697215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:02.702382 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 49/99] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:02.737518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be82b39-35b5-496a-9025-8af8257c51c8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:02.864800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be82b39-35b5-496a-9025-8af8257c51c8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:02.915882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6434e-fcd4-435a-b83a-b915acc460a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:02.917211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6434e-fcd4-435a-b83a-b915acc460a3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:02.917729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6434e-fcd4-435a-b83a-b915acc460a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:02.918209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6434e-fcd4-435a-b83a-b915acc460a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:02.935524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6434e-fcd4-435a-b83a-b915acc460a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2ce6434e-fcd4-435a-b83a-b915acc460a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:02.947818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:02.948675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:02.949437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:02.956064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 51/100] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:02.964412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87a099-8d68-46d1-b286-98c4ea143de0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:03.121358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87a099-8d68-46d1-b286-98c4ea143de0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:03.162361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:03.163047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.163363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.163651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.164412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:03.164727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:03.166142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:03.175002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 50/101] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:03.180806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6434e-fcd4-435a-b83a-b915acc460a3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:03.212809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:03.315035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6434e-fcd4-435a-b83a-b915acc460a3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:03.360317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1e52c-3cda-47d5-a7dc-e2672651310e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:03.361589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1e52c-3cda-47d5-a7dc-e2672651310e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.362084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1e52c-3cda-47d5-a7dc-e2672651310e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.362577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1e52c-3cda-47d5-a7dc-e2672651310e None admin] PATH_INFO: `/v3/users/4535c49340a94959bde8c258d072216d` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.388045 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1e52c-3cda-47d5-a7dc-e2672651310e None admin] RBAC: Authorizing `identity:get_user(user_id=4535c49340a94959bde8c258d072216d)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:03.389902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1e52c-3cda-47d5-a7dc-e2672651310e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:03.392443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 52/102] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:03 2022] GET /identity/v3/users/4535c49340a94959bde8c258d072216d => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:03.397627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:03.481475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:03.528130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202787d1-de94-467e-b764-20afc59a0a0f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:03.534197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225271f8-af5d-4339-826d-c32443ec2706 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.534770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225271f8-af5d-4339-826d-c32443ec2706 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.535247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225271f8-af5d-4339-826d-c32443ec2706 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.536937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 53/103] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:03.542136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcb3ea-53b1-4e45-aa91-ba155ccd65be None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.542686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcb3ea-53b1-4e45-aa91-ba155ccd65be None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.543176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcb3ea-53b1-4e45-aa91-ba155ccd65be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.572305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3bbcb3ea-53b1-4e45-aa91-ba155ccd65be None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:03.574017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3bbcb3ea-53b1-4e45-aa91-ba155ccd65be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:03.589385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502b9b6-eb28-41ad-abdd-0e538684fca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:03.590046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b9b6-eb28-41ad-abdd-0e538684fca6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.590339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b9b6-eb28-41ad-abdd-0e538684fca6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.590621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502b9b6-eb28-41ad-abdd-0e538684fca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.711993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 54/104] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:03.729170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502b9b6-eb28-41ad-abdd-0e538684fca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:03.732354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a6651-626d-4425-937b-322e6d451128 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.732865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a6651-626d-4425-937b-322e6d451128 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.733321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a6651-626d-4425-937b-322e6d451128 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.734879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 55/105] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:03.735406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502b9b6-eb28-41ad-abdd-0e538684fca6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:03.739820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c87e5d-4b11-4e95-af94-70f6da7f7027 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.740406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c87e5d-4b11-4e95-af94-70f6da7f7027 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.740875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c87e5d-4b11-4e95-af94-70f6da7f7027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.775356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-64c87e5d-4b11-4e95-af94-70f6da7f7027 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:03.777057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-64c87e5d-4b11-4e95-af94-70f6da7f7027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:03.780240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:03.849087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:03.879857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 56/106] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:03.881190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c87e5d-4b11-4e95-af94-70f6da7f7027 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:03.913646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 51/107] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:03.919819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bc805-eae9-4c31-b225-ccfa6f59357e None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.920179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bc805-eae9-4c31-b225-ccfa6f59357e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.920528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bc805-eae9-4c31-b225-ccfa6f59357e None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.921962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 52/108] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:03.922850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bc805-eae9-4c31-b225-ccfa6f59357e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.017744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c87e5d-4b11-4e95-af94-70f6da7f7027 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.042540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bc805-eae9-4c31-b225-ccfa6f59357e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.062698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a223a5-69f4-47c1-b94a-ccae636a6e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.063789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a223a5-69f4-47c1-b94a-ccae636a6e99 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.064226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a223a5-69f4-47c1-b94a-ccae636a6e99 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.064671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a223a5-69f4-47c1-b94a-ccae636a6e99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.066428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a223a5-69f4-47c1-b94a-ccae636a6e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:04.068593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a223a5-69f4-47c1-b94a-ccae636a6e99 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:04.071481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 57/109] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:04.072766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a223a5-69f4-47c1-b94a-ccae636a6e99 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.080828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038aab7-0a17-4506-937e-9344af570aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.081813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038aab7-0a17-4506-937e-9344af570aa4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.082205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038aab7-0a17-4506-937e-9344af570aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.082663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038aab7-0a17-4506-937e-9344af570aa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.084226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:04.096134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038aab7-0a17-4506-937e-9344af570aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a038aab7-0a17-4506-937e-9344af570aa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:04.104252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:04.105335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:04.106293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:04.111560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 53/110] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:32:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:04.114742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484114d3-799f-46ad-9931-8d0fabee2e55 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.115693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484114d3-799f-46ad-9931-8d0fabee2e55 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.116123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484114d3-799f-46ad-9931-8d0fabee2e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.133437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-484114d3-799f-46ad-9931-8d0fabee2e55 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:04.134956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-484114d3-799f-46ad-9931-8d0fabee2e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:04.154380 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:04.193131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a223a5-69f4-47c1-b94a-ccae636a6e99 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.228050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 54/111] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:04.229102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484114d3-799f-46ad-9931-8d0fabee2e55 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.239573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db492521-621d-4339-aefa-b19850f6aa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.241458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db492521-621d-4339-aefa-b19850f6aa71 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.241919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db492521-621d-4339-aefa-b19850f6aa71 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.242369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db492521-621d-4339-aefa-b19850f6aa71 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.243944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db492521-621d-4339-aefa-b19850f6aa71 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:04.245758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db492521-621d-4339-aefa-b19850f6aa71 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:04.247836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 58/112] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:04 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:04.248855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db492521-621d-4339-aefa-b19850f6aa71 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.329056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:04.336929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484114d3-799f-46ad-9931-8d0fabee2e55 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.355553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db492521-621d-4339-aefa-b19850f6aa71 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.373757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d266d23-f35a-4622-b152-82d73c928601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.374454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266d23-f35a-4622-b152-82d73c928601 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.374718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266d23-f35a-4622-b152-82d73c928601 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.375003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d266d23-f35a-4622-b152-82d73c928601 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.375836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d266d23-f35a-4622-b152-82d73c928601 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:04.377302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d266d23-f35a-4622-b152-82d73c928601 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:04.395973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e4b1-22a7-4777-9e62-193351c70a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.397305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e4b1-22a7-4777-9e62-193351c70a3c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.397655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e4b1-22a7-4777-9e62-193351c70a3c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.398310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e4b1-22a7-4777-9e62-193351c70a3c None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.399156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e4b1-22a7-4777-9e62-193351c70a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:04.399554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e4b1-22a7-4777-9e62-193351c70a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:04.401035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e4b1-22a7-4777-9e62-193351c70a3c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:04.410372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 59/113] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:04.414727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fd1ce-2694-4980-a752-4540f4efe5a8 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.414727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fd1ce-2694-4980-a752-4540f4efe5a8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.414727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fd1ce-2694-4980-a752-4540f4efe5a8 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.414727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 60/114] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:04.414988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fd1ce-2694-4980-a752-4540f4efe5a8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.468242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 55/115] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:32:04 2022] POST /identity/v3/projects => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:04.476699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d266d23-f35a-4622-b152-82d73c928601 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.488783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:04.532365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fd1ce-2694-4980-a752-4540f4efe5a8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.570731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.571796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.572285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.572741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.573801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] RBAC: Adding query filter params (role.id=3555f68f0fca46018de9022f06dcdc90, scope.project.id=b7fddf3939de4bb99741de8e3307565f, user.id=4535c49340a94959bde8c258d072216d) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:04.574321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:04.576269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:04.582823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 61/116] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:04 2022] GET /identity/v3/role_assignments?user.id=4535c49340a94959bde8c258d072216d&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=3555f68f0fca46018de9022f06dcdc90 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:04.584775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.585733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:04.629540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d266d23-f35a-4622-b152-82d73c928601 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.683897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68caed-bdec-42e0-8e02-c434854a0763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.685513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68caed-bdec-42e0-8e02-c434854a0763 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.685964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68caed-bdec-42e0-8e02-c434854a0763 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.686793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68caed-bdec-42e0-8e02-c434854a0763 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.709136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68caed-bdec-42e0-8e02-c434854a0763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-df68caed-bdec-42e0-8e02-c434854a0763 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:04.719725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:04.720626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:04.721397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:04.721961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 56/117] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:04.723801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b488f59-7cf0-4c25-bab2-66c359e6bad4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.723801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b488f59-7cf0-4c25-bab2-66c359e6bad4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.723801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b488f59-7cf0-4c25-bab2-66c359e6bad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.743988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5b488f59-7cf0-4c25-bab2-66c359e6bad4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:04.744441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30496c99-0023-484b-a89d-004b6f20b5da None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.745224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5b488f59-7cf0-4c25-bab2-66c359e6bad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:04.785302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe98d8-10bd-4faf-9e6c-fdc0a6e090c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.786203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe98d8-10bd-4faf-9e6c-fdc0a6e090c1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.786493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe98d8-10bd-4faf-9e6c-fdc0a6e090c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.787352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe98d8-10bd-4faf-9e6c-fdc0a6e090c1 None admin] PATH_INFO: `/v3/users/fb07001f45bb40c8b8c84b7d67d1c304` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.789319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe98d8-10bd-4faf-9e6c-fdc0a6e090c1 None admin] RBAC: Authorizing `identity:get_user(user_id=fb07001f45bb40c8b8c84b7d67d1c304)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:04.790969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe98d8-10bd-4faf-9e6c-fdc0a6e090c1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:04.792959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 62/118] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:04 2022] GET /identity/v3/users/fb07001f45bb40c8b8c84b7d67d1c304 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:04.793845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe98d8-10bd-4faf-9e6c-fdc0a6e090c1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.813425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:04.822554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 57/119] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:04.823562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b488f59-7cf0-4c25-bab2-66c359e6bad4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:04.875580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:04.912455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe98d8-10bd-4faf-9e6c-fdc0a6e090c1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.947765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b488f59-7cf0-4c25-bab2-66c359e6bad4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:04.949761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5676b2-f95b-4ae7-af1a-bf2f57ec099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:04.950500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5676b2-f95b-4ae7-af1a-bf2f57ec099a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.950914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5676b2-f95b-4ae7-af1a-bf2f57ec099a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.951353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5676b2-f95b-4ae7-af1a-bf2f57ec099a None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.952309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5676b2-f95b-4ae7-af1a-bf2f57ec099a None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:04.952620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5676b2-f95b-4ae7-af1a-bf2f57ec099a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:04.954140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5676b2-f95b-4ae7-af1a-bf2f57ec099a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:04.963375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 63/120] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:04.964636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac10049-4f1f-4ad6-82c6-ac5a8e45954a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:04.964930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac10049-4f1f-4ad6-82c6-ac5a8e45954a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:04.965298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac10049-4f1f-4ad6-82c6-ac5a8e45954a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:04.966574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 64/121] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:04.967280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac10049-4f1f-4ad6-82c6-ac5a8e45954a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.028272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289dc5f-b830-4565-ba2c-16047e381b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.029198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289dc5f-b830-4565-ba2c-16047e381b7c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.029485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289dc5f-b830-4565-ba2c-16047e381b7c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.029769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289dc5f-b830-4565-ba2c-16047e381b7c None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.031073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289dc5f-b830-4565-ba2c-16047e381b7c None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:05.032619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289dc5f-b830-4565-ba2c-16047e381b7c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:05.034761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 58/122] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:04 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:05.035792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289dc5f-b830-4565-ba2c-16047e381b7c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.104691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:05.120808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac10049-4f1f-4ad6-82c6-ac5a8e45954a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:05.148368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289dc5f-b830-4565-ba2c-16047e381b7c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:05.158568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:05.171945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bf6c6-c83e-42b8-b622-868b282fe519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.172774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bf6c6-c83e-42b8-b622-868b282fe519 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.173132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bf6c6-c83e-42b8-b622-868b282fe519 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.173416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bf6c6-c83e-42b8-b622-868b282fe519 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.184772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bf6c6-c83e-42b8-b622-868b282fe519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:05.189546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9217f-8f39-4f9b-ab5e-c696c5dbc2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.190425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9217f-8f39-4f9b-ab5e-c696c5dbc2de None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.190712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9217f-8f39-4f9b-ab5e-c696c5dbc2de None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.190992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9217f-8f39-4f9b-ab5e-c696c5dbc2de None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.192461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9217f-8f39-4f9b-ab5e-c696c5dbc2de None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-e63bf6c6-c83e-42b8-b622-868b282fe519 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:05.194184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:05.195925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:05.197590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:05.198831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9217f-8f39-4f9b-ab5e-c696c5dbc2de None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:05.198959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 65/123] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:05.203339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e3afd-e2d8-410b-b1cd-d964a370eae6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.203339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e3afd-e2d8-410b-b1cd-d964a370eae6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.203339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e3afd-e2d8-410b-b1cd-d964a370eae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.203730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 59/124] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:05 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:05.203730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9217f-8f39-4f9b-ab5e-c696c5dbc2de None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.220754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e10e3afd-e2d8-410b-b1cd-d964a370eae6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:05.222197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e10e3afd-e2d8-410b-b1cd-d964a370eae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:05.300274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 66/125] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:05.301125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e3afd-e2d8-410b-b1cd-d964a370eae6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.345909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9217f-8f39-4f9b-ab5e-c696c5dbc2de None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:05.378947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:05.386445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2585b56-abe6-4300-850b-5c0431e57512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.387519 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2585b56-abe6-4300-850b-5c0431e57512 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.387867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2585b56-abe6-4300-850b-5c0431e57512 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.388205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2585b56-abe6-4300-850b-5c0431e57512 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f/users/fb07001f45bb40c8b8c84b7d67d1c304/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.390868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2585b56-abe6-4300-850b-5c0431e57512 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7fddf3939de4bb99741de8e3307565f, user_id=fb07001f45bb40c8b8c84b7d67d1c304, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:05.393942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2585b56-abe6-4300-850b-5c0431e57512 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:05.425476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e3afd-e2d8-410b-b1cd-d964a370eae6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:05.447678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:05.462096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7822a9b-b7b3-4f57-a09b-a7aaf887d641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.463073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7822a9b-b7b3-4f57-a09b-a7aaf887d641 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.463402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7822a9b-b7b3-4f57-a09b-a7aaf887d641 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.463674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7822a9b-b7b3-4f57-a09b-a7aaf887d641 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.464847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7822a9b-b7b3-4f57-a09b-a7aaf887d641 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:05.466310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7822a9b-b7b3-4f57-a09b-a7aaf887d641 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:05.468320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 67/126] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:05 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:05.468888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7822a9b-b7b3-4f57-a09b-a7aaf887d641 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.476456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 60/127] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:05 2022] PUT /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f/users/fb07001f45bb40c8b8c84b7d67d1c304/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:05.477623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2585b56-abe6-4300-850b-5c0431e57512 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.613821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2585b56-abe6-4300-850b-5c0431e57512 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:05.657279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454393df-8ce3-4d72-9f85-b17bf60d0246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.658083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454393df-8ce3-4d72-9f85-b17bf60d0246 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.658376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454393df-8ce3-4d72-9f85-b17bf60d0246 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.658716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454393df-8ce3-4d72-9f85-b17bf60d0246 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.668350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7822a9b-b7b3-4f57-a09b-a7aaf887d641 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:05.671437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454393df-8ce3-4d72-9f85-b17bf60d0246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-454393df-8ce3-4d72-9f85-b17bf60d0246 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:05.679169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:05.680159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:05.680908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:05.681476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 61/128] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:05.682740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454393df-8ce3-4d72-9f85-b17bf60d0246 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.711621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.712805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.713359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.713719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.714644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:05.714964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:05.717144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:05.727810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 68/129] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:05.728619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.751603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:05.822017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454393df-8ce3-4d72-9f85-b17bf60d0246 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:05.836393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:05.860776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ad0bc-bced-4e35-be33-a3a1059db665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.862099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ad0bc-bced-4e35-be33-a3a1059db665 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.862636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ad0bc-bced-4e35-be33-a3a1059db665 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.863113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ad0bc-bced-4e35-be33-a3a1059db665 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f/users/c956159832a6436bbf4c4a3517bd6170/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.866055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ad0bc-bced-4e35-be33-a3a1059db665 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7fddf3939de4bb99741de8e3307565f, user_id=c956159832a6436bbf4c4a3517bd6170, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:05.867848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ad0bc-bced-4e35-be33-a3a1059db665 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:05.916008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc5850-34fe-455c-b90d-54bd7445e2b7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:05.952927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:05.953612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:05.953907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:05.954193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:05.954948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:05.955264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:05.956710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:05.966156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 69/130] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:05.967066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:05.993436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 62/131] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:05 2022] PUT /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f/users/c956159832a6436bbf4c4a3517bd6170/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:05.994679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ad0bc-bced-4e35-be33-a3a1059db665 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:06.219332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10524cf2-017a-47a7-a10b-a294b8081ffe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:06.247106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ad0bc-bced-4e35-be33-a3a1059db665 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:06.254255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:06.262044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:06.291604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb8a7a-2b27-4cae-97ee-e84736156333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:06.292813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb8a7a-2b27-4cae-97ee-e84736156333 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:06.293194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb8a7a-2b27-4cae-97ee-e84736156333 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:06.293481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb8a7a-2b27-4cae-97ee-e84736156333 None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:06.294689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb8a7a-2b27-4cae-97ee-e84736156333 None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:06.296257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb8a7a-2b27-4cae-97ee-e84736156333 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:06.296766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255eb7-3c86-4cd3-b34c-c539bc84d023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:06.298849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 63/132] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:05 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:06.299072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255eb7-3c86-4cd3-b34c-c539bc84d023 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:06.299577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255eb7-3c86-4cd3-b34c-c539bc84d023 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:06.300029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7255eb7-3c86-4cd3-b34c-c539bc84d023 None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:06.301479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7255eb7-3c86-4cd3-b34c-c539bc84d023 None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:06.303287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7255eb7-3c86-4cd3-b34c-c539bc84d023 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:06.303690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb8a7a-2b27-4cae-97ee-e84736156333 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:06.307613 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 70/133] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:05 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:06.316325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255eb7-3c86-4cd3-b34c-c539bc84d023 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:06.426683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb8a7a-2b27-4cae-97ee-e84736156333 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:06.466801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6fb7b-aa91-4ae4-a917-759921efdb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:06.467758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6fb7b-aa91-4ae4-a917-759921efdb1b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:06.468046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6fb7b-aa91-4ae4-a917-759921efdb1b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:06.468330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6fb7b-aa91-4ae4-a917-759921efdb1b None admin] PATH_INFO: `/v3/projects/d0157c96b92744a4b2956b1ef2c7d110` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:06.473217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6fb7b-aa91-4ae4-a917-759921efdb1b None admin] RBAC: Authorizing `identity:get_project(project_id=d0157c96b92744a4b2956b1ef2c7d110)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:06.473217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6fb7b-aa91-4ae4-a917-759921efdb1b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:06.473217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 64/134] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:06 2022] GET /identity/v3/projects/d0157c96b92744a4b2956b1ef2c7d110 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:06.479004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6fb7b-aa91-4ae4-a917-759921efdb1b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:06.532899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:06.569518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7255eb7-3c86-4cd3-b34c-c539bc84d023 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:06.596458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6fb7b-aa91-4ae4-a917-759921efdb1b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:06.636336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc1746-e57e-4f76-9beb-369cbb626539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:06.637741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc1746-e57e-4f76-9beb-369cbb626539 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:06.638227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc1746-e57e-4f76-9beb-369cbb626539 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:06.638700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc1746-e57e-4f76-9beb-369cbb626539 None admin] PATH_INFO: `/v3/projects/d0157c96b92744a4b2956b1ef2c7d110/users/c956159832a6436bbf4c4a3517bd6170/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:06.641490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc1746-e57e-4f76-9beb-369cbb626539 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0157c96b92744a4b2956b1ef2c7d110, user_id=c956159832a6436bbf4c4a3517bd6170, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:06.643354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc1746-e57e-4f76-9beb-369cbb626539 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:06.655979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e7079-0f39-48cc-bb09-c12d51332e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:06.656865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7079-0f39-48cc-bb09-c12d51332e37 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:06.657155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7079-0f39-48cc-bb09-c12d51332e37 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:06.657437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7079-0f39-48cc-bb09-c12d51332e37 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:06.665489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e7079-0f39-48cc-bb09-c12d51332e37 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:06.665489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e7079-0f39-48cc-bb09-c12d51332e37 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:06.665489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 71/135] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:06 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:06.668915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e7079-0f39-48cc-bb09-c12d51332e37 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:06.713288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 65/136] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:06 2022] PUT /identity/v3/projects/d0157c96b92744a4b2956b1ef2c7d110/users/c956159832a6436bbf4c4a3517bd6170/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:06.744059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:06.804175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e7079-0f39-48cc-bb09-c12d51332e37 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:06.842602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a64e62-4c23-4155-9671-e3ccd271ca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:06.844312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a64e62-4c23-4155-9671-e3ccd271ca5f None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:06.844794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a64e62-4c23-4155-9671-e3ccd271ca5f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:06.845288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a64e62-4c23-4155-9671-e3ccd271ca5f None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f/users/c956159832a6436bbf4c4a3517bd6170/roles/9a22f6f23fb64da488d982c17ed565ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:06.847782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a64e62-4c23-4155-9671-e3ccd271ca5f None admin] RBAC: Authorizing `identity:create_grant(project_id=b7fddf3939de4bb99741de8e3307565f, user_id=c956159832a6436bbf4c4a3517bd6170, role_id=9a22f6f23fb64da488d982c17ed565ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:06.850514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a64e62-4c23-4155-9671-e3ccd271ca5f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:06.920406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 72/137] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:06 2022] PUT /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f/users/c956159832a6436bbf4c4a3517bd6170/roles/9a22f6f23fb64da488d982c17ed565ce => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:32:07.059472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be810-4c4a-4ff5-adc2-0cb6b69f00f5 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:07.060194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be810-4c4a-4ff5-adc2-0cb6b69f00f5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:07.060685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4be810-4c4a-4ff5-adc2-0cb6b69f00f5 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:07.062669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 66/138] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:07.068251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afab31-9cea-4945-bb2f-9df8c78e2633 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:07.068850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afab31-9cea-4945-bb2f-9df8c78e2633 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:07.069342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afab31-9cea-4945-bb2f-9df8c78e2633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:07.090572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-65afab31-9cea-4945-bb2f-9df8c78e2633 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:07.092151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-65afab31-9cea-4945-bb2f-9df8c78e2633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:07.169049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 73/139] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:07.191113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4be810-4c4a-4ff5-adc2-0cb6b69f00f5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:07.257094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53529-97af-4aec-aebb-86bc7cf9bcdb None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:07.257916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53529-97af-4aec-aebb-86bc7cf9bcdb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:07.258770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53529-97af-4aec-aebb-86bc7cf9bcdb None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:07.261108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 74/140] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:07.268064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004b5d3-944c-4bc3-acff-478953541998 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:07.268736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004b5d3-944c-4bc3-acff-478953541998 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:07.269208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004b5d3-944c-4bc3-acff-478953541998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:07.295556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c004b5d3-944c-4bc3-acff-478953541998 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:07.297133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c004b5d3-944c-4bc3-acff-478953541998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:07.316970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4be810-4c4a-4ff5-adc2-0cb6b69f00f5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:07.326903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:07.364559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:07.383995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52fa70-63a5-414c-b7c7-b4832a0da6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:07.384821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52fa70-63a5-414c-b7c7-b4832a0da6e8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:07.385111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52fa70-63a5-414c-b7c7-b4832a0da6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:07.385396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52fa70-63a5-414c-b7c7-b4832a0da6e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:07.415334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52fa70-63a5-414c-b7c7-b4832a0da6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-cd52fa70-63a5-414c-b7c7-b4832a0da6e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:07.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:07.428851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:07.429597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 67/141] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:07.437336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52fa70-63a5-414c-b7c7-b4832a0da6e8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:07.439261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 75/142] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:07.466385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004b5d3-944c-4bc3-acff-478953541998 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:07.571796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004b5d3-944c-4bc3-acff-478953541998 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:07.612189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406785ef-ce3f-45a2-9967-e051d5f80087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:07.613003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406785ef-ce3f-45a2-9967-e051d5f80087 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:07.613289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406785ef-ce3f-45a2-9967-e051d5f80087 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:07.613567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406785ef-ce3f-45a2-9967-e051d5f80087 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:07.615783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406785ef-ce3f-45a2-9967-e051d5f80087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:07.617568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406785ef-ce3f-45a2-9967-e051d5f80087 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:07.619804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 76/143] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:07.626158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406785ef-ce3f-45a2-9967-e051d5f80087 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:07.670987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:07.683177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52fa70-63a5-414c-b7c7-b4832a0da6e8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:07.728068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:07.728797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:07.729058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:07.729335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:07.730107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:07.730419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:07.731877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:07.754200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 68/144] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:07.763537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:07.816131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406785ef-ce3f-45a2-9967-e051d5f80087 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:07.844176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:07.906331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d21b30-2f16-489f-9d28-db82a22fe4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:07.907541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d21b30-2f16-489f-9d28-db82a22fe4d1 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:07.908178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d21b30-2f16-489f-9d28-db82a22fe4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:07.908601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d21b30-2f16-489f-9d28-db82a22fe4d1 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:07.909785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d21b30-2f16-489f-9d28-db82a22fe4d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:07.911620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d21b30-2f16-489f-9d28-db82a22fe4d1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:07.994059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984cb92-e1c3-4dbe-ac6c-ff5397efb1a8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:08.019512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 77/145] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:07 2022] POST /identity/v3/users => generated 283 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:32:08.041628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32c508-992c-4734-b080-a64ff7f605ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:08.042819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32c508-992c-4734-b080-a64ff7f605ca None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:08.043283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32c508-992c-4734-b080-a64ff7f605ca None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:08.043786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32c508-992c-4734-b080-a64ff7f605ca None admin] PATH_INFO: `/v3/users/4535c49340a94959bde8c258d072216d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:08.047761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32c508-992c-4734-b080-a64ff7f605ca None admin] RBAC: Authorizing `identity:get_user(user_id=4535c49340a94959bde8c258d072216d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:08.049502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32c508-992c-4734-b080-a64ff7f605ca None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:08.051518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 69/146] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:07 2022] GET /identity/v3/users/4535c49340a94959bde8c258d072216d => generated 289 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:08.057227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d21b30-2f16-489f-9d28-db82a22fe4d1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:08.195573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:08.204004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d21b30-2f16-489f-9d28-db82a22fe4d1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:08.253025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22c3d4-29be-4a61-8509-7bc15a2cd9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:08.253916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22c3d4-29be-4a61-8509-7bc15a2cd9f6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:08.254202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22c3d4-29be-4a61-8509-7bc15a2cd9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:08.254481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22c3d4-29be-4a61-8509-7bc15a2cd9f6 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:08.256252 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22c3d4-29be-4a61-8509-7bc15a2cd9f6 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:08.257752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22c3d4-29be-4a61-8509-7bc15a2cd9f6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:08.259842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 78/147] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:08 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:08.265464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32c508-992c-4734-b080-a64ff7f605ca None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:08.357335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f0714-faec-4d6f-8a9b-ef0b2e6494bd None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:08.357623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f0714-faec-4d6f-8a9b-ef0b2e6494bd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:08.357886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f0714-faec-4d6f-8a9b-ef0b2e6494bd None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:08.359454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 79/148] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:08.365053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce39a-8287-4a7d-9f82-e9c5f45619cb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:08.366350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce39a-8287-4a7d-9f82-e9c5f45619cb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:08.367004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce39a-8287-4a7d-9f82-e9c5f45619cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:08.396460 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f24ce39a-8287-4a7d-9f82-e9c5f45619cb None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:08.398273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f24ce39a-8287-4a7d-9f82-e9c5f45619cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:08.441973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:08.453291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32c508-992c-4734-b080-a64ff7f605ca None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:08.490428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e593500-9a55-4b01-9e68-0122d2a13596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:08.491365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 80/149] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:08.491676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e593500-9a55-4b01-9e68-0122d2a13596 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:08.491966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e593500-9a55-4b01-9e68-0122d2a13596 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:08.492250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e593500-9a55-4b01-9e68-0122d2a13596 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f/users/4535c49340a94959bde8c258d072216d/roles/3555f68f0fca46018de9022f06dcdc90` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:08.496845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e593500-9a55-4b01-9e68-0122d2a13596 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7fddf3939de4bb99741de8e3307565f, user_id=4535c49340a94959bde8c258d072216d, role_id=3555f68f0fca46018de9022f06dcdc90)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:08.498317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e593500-9a55-4b01-9e68-0122d2a13596 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:08.520202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ce39a-8287-4a7d-9f82-e9c5f45619cb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:08.577794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 70/150] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:08 2022] PUT /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f/users/4535c49340a94959bde8c258d072216d/roles/3555f68f0fca46018de9022f06dcdc90 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:08.623417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:08.634105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ce39a-8287-4a7d-9f82-e9c5f45619cb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:08.673175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725e625-521d-41ee-b49f-a95147bc4eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:08.673992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e625-521d-41ee-b49f-a95147bc4eee None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:08.674286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e625-521d-41ee-b49f-a95147bc4eee None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:08.674571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e625-521d-41ee-b49f-a95147bc4eee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:08.686358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725e625-521d-41ee-b49f-a95147bc4eee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-f725e625-521d-41ee-b49f-a95147bc4eee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:08.695142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:08.696047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:08.696828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 81/151] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:08.703946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e593500-9a55-4b01-9e68-0122d2a13596 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:08.821021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e593500-9a55-4b01-9e68-0122d2a13596 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:08.860667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:08.861481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:08.861781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:08.862071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:08.862845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:08.863164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:08.864622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:08.875577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 71/152] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:08.881457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725e625-521d-41ee-b49f-a95147bc4eee None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:08.991118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725e625-521d-41ee-b49f-a95147bc4eee None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:09.027282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2150a2-8e39-4737-a134-46c962fd5907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:09.028203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2150a2-8e39-4737-a134-46c962fd5907 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.028494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2150a2-8e39-4737-a134-46c962fd5907 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.028773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2150a2-8e39-4737-a134-46c962fd5907 None admin] PATH_INFO: `/v3/users/fb07001f45bb40c8b8c84b7d67d1c304` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.030049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2150a2-8e39-4737-a134-46c962fd5907 None admin] RBAC: Authorizing `identity:get_user(user_id=fb07001f45bb40c8b8c84b7d67d1c304)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:09.031639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2150a2-8e39-4737-a134-46c962fd5907 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:09.033559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 82/153] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:08 2022] GET /identity/v3/users/fb07001f45bb40c8b8c84b7d67d1c304 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:09.039718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:09.050864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afd1a-662b-4ddc-845b-4301950caf36 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.051159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afd1a-662b-4ddc-845b-4301950caf36 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.051478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afd1a-662b-4ddc-845b-4301950caf36 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.053021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 83/154] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:09.060378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20604133-7401-449f-929a-12b0980cf019 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.060666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20604133-7401-449f-929a-12b0980cf019 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.060951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20604133-7401-449f-929a-12b0980cf019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.072905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:09.094258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-20604133-7401-449f-929a-12b0980cf019 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:09.096476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-20604133-7401-449f-929a-12b0980cf019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:09.131207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:09.239534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 84/155] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:09.241538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669a62d-323c-4fd1-aae4-5e49fd130f85 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:09.286241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20604133-7401-449f-929a-12b0980cf019 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:09.298047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a23d6-be36-4b89-b8ce-cc08f2e419ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:09.298723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a23d6-be36-4b89-b8ce-cc08f2e419ea None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.299018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a23d6-be36-4b89-b8ce-cc08f2e419ea None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.299322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a23d6-be36-4b89-b8ce-cc08f2e419ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.411315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20604133-7401-449f-929a-12b0980cf019 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:09.444836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e11acd-fad9-47e9-a99e-b08ce328353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:09.445658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11acd-fad9-47e9-a99e-b08ce328353e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.445946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11acd-fad9-47e9-a99e-b08ce328353e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.446224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e11acd-fad9-47e9-a99e-b08ce328353e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.460243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e11acd-fad9-47e9-a99e-b08ce328353e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-f9e11acd-fad9-47e9-a99e-b08ce328353e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:09.468351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:09.469273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:09.470023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 85/156] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:09.472583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a23d6-be36-4b89-b8ce-cc08f2e419ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:09.474091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a23d6-be36-4b89-b8ce-cc08f2e419ea None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:09.476689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e11acd-fad9-47e9-a99e-b08ce328353e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:09.495766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:09.520425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:09.585874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e11acd-fad9-47e9-a99e-b08ce328353e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:09.621950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:09.622640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.622939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.623219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.624068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:09.624385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:09.625809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:09.634999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 86/157] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:09.640484 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:09.723850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:09.730126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 72/158] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:09.740987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a688238-c5b7-4435-9a60-010939938662 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.741283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a688238-c5b7-4435-9a60-010939938662 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.746242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a688238-c5b7-4435-9a60-010939938662 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.746242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 73/159] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:09.749071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7a3ef-75ec-4914-ac02-8440aef775ff None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.749480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7a3ef-75ec-4914-ac02-8440aef775ff None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.749921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7a3ef-75ec-4914-ac02-8440aef775ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.754321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453f1f1-eb8e-4870-995d-71cd48242d71 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:09.770148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a6c7a3ef-75ec-4914-ac02-8440aef775ff None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:09.771648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a6c7a3ef-75ec-4914-ac02-8440aef775ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:09.803349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646d1c4-e9bb-4554-b63f-6a58b4afacba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:09.804241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646d1c4-e9bb-4554-b63f-6a58b4afacba None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:09.804791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646d1c4-e9bb-4554-b63f-6a58b4afacba None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:09.805076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646d1c4-e9bb-4554-b63f-6a58b4afacba None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:09.807038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646d1c4-e9bb-4554-b63f-6a58b4afacba None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:09.809508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646d1c4-e9bb-4554-b63f-6a58b4afacba None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:09.812201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 87/160] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:09 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:09.813034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646d1c4-e9bb-4554-b63f-6a58b4afacba None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:09.865652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 74/161] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:09.866582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7a3ef-75ec-4914-ac02-8440aef775ff None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:09.954290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:09.958035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646d1c4-e9bb-4554-b63f-6a58b4afacba None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:10.005174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35686-47eb-4198-88d7-c44c898c1bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:10.006018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35686-47eb-4198-88d7-c44c898c1bfd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.006310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35686-47eb-4198-88d7-c44c898c1bfd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.006593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d35686-47eb-4198-88d7-c44c898c1bfd None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.008017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d35686-47eb-4198-88d7-c44c898c1bfd None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:10.009968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d35686-47eb-4198-88d7-c44c898c1bfd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:10.012817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 88/162] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:09 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:10.013852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35686-47eb-4198-88d7-c44c898c1bfd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.026390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:10.028187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7a3ef-75ec-4914-ac02-8440aef775ff None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:10.074202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff4979-2897-459e-a36a-ac1a73f7ddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:10.075134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff4979-2897-459e-a36a-ac1a73f7ddf5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.075604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff4979-2897-459e-a36a-ac1a73f7ddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.076030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff4979-2897-459e-a36a-ac1a73f7ddf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.175197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d35686-47eb-4198-88d7-c44c898c1bfd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:10.196940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff4979-2897-459e-a36a-ac1a73f7ddf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:10.198875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff4979-2897-459e-a36a-ac1a73f7ddf5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:10.287071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:10.296023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093578bd-1ec8-4027-ae3c-a3b87b4b1f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:10.296840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093578bd-1ec8-4027-ae3c-a3b87b4b1f67 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.297128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093578bd-1ec8-4027-ae3c-a3b87b4b1f67 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.297403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093578bd-1ec8-4027-ae3c-a3b87b4b1f67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.351943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093578bd-1ec8-4027-ae3c-a3b87b4b1f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-093578bd-1ec8-4027-ae3c-a3b87b4b1f67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:10.359658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:10.360681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:10.361618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:10.362249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 75/163] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:10.363173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff4979-2897-459e-a36a-ac1a73f7ddf5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.364074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 89/164] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:10.368456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093578bd-1ec8-4027-ae3c-a3b87b4b1f67 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.479654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:10.541524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093578bd-1ec8-4027-ae3c-a3b87b4b1f67 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:10.561948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff4979-2897-459e-a36a-ac1a73f7ddf5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:10.586351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fb262-b6f4-4620-9476-b70a9e9eb77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:10.587606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fb262-b6f4-4620-9476-b70a9e9eb77d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.588122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fb262-b6f4-4620-9476-b70a9e9eb77d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.588639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fb262-b6f4-4620-9476-b70a9e9eb77d None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.590073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fb262-b6f4-4620-9476-b70a9e9eb77d None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:10.591943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fb262-b6f4-4620-9476-b70a9e9eb77d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:10.593897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 90/165] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:10.595037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fb262-b6f4-4620-9476-b70a9e9eb77d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.618714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427590f-91b4-42d4-884e-541ada553d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:10.619400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427590f-91b4-42d4-884e-541ada553d3a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.619689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427590f-91b4-42d4-884e-541ada553d3a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.619968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427590f-91b4-42d4-884e-541ada553d3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.620797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427590f-91b4-42d4-884e-541ada553d3a None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=b7fddf3939de4bb99741de8e3307565f, user.id=fb07001f45bb40c8b8c84b7d67d1c304) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:10.621111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427590f-91b4-42d4-884e-541ada553d3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:10.622550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427590f-91b4-42d4-884e-541ada553d3a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:10.636345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 76/166] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/role_assignments?user.id=fb07001f45bb40c8b8c84b7d67d1c304&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:10.637833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3760e-d619-40d8-a2de-685c7791d6f2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.638285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3760e-d619-40d8-a2de-685c7791d6f2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.638700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3760e-d619-40d8-a2de-685c7791d6f2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.644747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 77/167] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:10.645884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3760e-d619-40d8-a2de-685c7791d6f2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.670643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:10.710557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fb262-b6f4-4620-9476-b70a9e9eb77d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:10.749527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8264-6d0a-45ff-8637-716ea47f3674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:10.750556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8264-6d0a-45ff-8637-716ea47f3674 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.751065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8264-6d0a-45ff-8637-716ea47f3674 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.751688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8264-6d0a-45ff-8637-716ea47f3674 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.752654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8264-6d0a-45ff-8637-716ea47f3674 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:10.753202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8264-6d0a-45ff-8637-716ea47f3674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:10.754856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8264-6d0a-45ff-8637-716ea47f3674 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:10.765399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 91/168] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:10.770121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:10.771927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f203e-185f-4705-9e69-3c255230d6eb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.772538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f203e-185f-4705-9e69-3c255230d6eb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.773025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f203e-185f-4705-9e69-3c255230d6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.792864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b76f203e-185f-4705-9e69-3c255230d6eb None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:10.794401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b76f203e-185f-4705-9e69-3c255230d6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:10.817200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3760e-d619-40d8-a2de-685c7791d6f2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:10.872161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 92/169] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:10.873563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f203e-185f-4705-9e69-3c255230d6eb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.880016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:10.880729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.881021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.881302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.882149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] RBAC: Adding query filter params (role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=b7fddf3939de4bb99741de8e3307565f, user.id=c956159832a6436bbf4c4a3517bd6170) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:10.882464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:10.883952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:10.901353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 78/170] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/role_assignments?user.id=c956159832a6436bbf4c4a3517bd6170&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:10.934729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.988786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f203e-185f-4705-9e69-3c255230d6eb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.022434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:11.031977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438c27f-2f09-4620-9234-9ebe70353626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.033570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438c27f-2f09-4620-9234-9ebe70353626 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.034076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438c27f-2f09-4620-9234-9ebe70353626 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.034545 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438c27f-2f09-4620-9234-9ebe70353626 None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.035989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438c27f-2f09-4620-9234-9ebe70353626 None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.038111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438c27f-2f09-4620-9234-9ebe70353626 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.040562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 93/171] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:11.047695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438c27f-2f09-4620-9234-9ebe70353626 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.167325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438c27f-2f09-4620-9234-9ebe70353626 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.206586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44975c97-e9a1-4a15-8a13-74e8646281cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.207572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44975c97-e9a1-4a15-8a13-74e8646281cc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.208111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44975c97-e9a1-4a15-8a13-74e8646281cc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.208617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44975c97-e9a1-4a15-8a13-74e8646281cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.210602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:11.287679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:11.295881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44975c97-e9a1-4a15-8a13-74e8646281cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.297951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44975c97-e9a1-4a15-8a13-74e8646281cc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.315322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a1096-1f5a-4ec6-9498-8f3f49e29635 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.428233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae5cc2-cd3e-4d4f-bf8e-6a0ca966bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.429384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5cc2-cd3e-4d4f-bf8e-6a0ca966bd04 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.429852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5cc2-cd3e-4d4f-bf8e-6a0ca966bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.430332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5cc2-cd3e-4d4f-bf8e-6a0ca966bd04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.431356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 94/172] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 15:32:11.432841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44975c97-e9a1-4a15-8a13-74e8646281cc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.448407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae5cc2-cd3e-4d4f-bf8e-6a0ca966bd04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-eeae5cc2-cd3e-4d4f-bf8e-6a0ca966bd04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:11.456927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:11.457798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:11.458555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:11.465264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 79/173] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 531 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:11.470691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae5cc2-cd3e-4d4f-bf8e-6a0ca966bd04 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.546966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44975c97-e9a1-4a15-8a13-74e8646281cc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.559730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:11.589478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec429f5d-5afb-4faf-9d52-534291b94287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.590359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec429f5d-5afb-4faf-9d52-534291b94287 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.590644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec429f5d-5afb-4faf-9d52-534291b94287 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.590933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec429f5d-5afb-4faf-9d52-534291b94287 None admin] PATH_INFO: `/v3/projects/d0157c96b92744a4b2956b1ef2c7d110` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.593476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec429f5d-5afb-4faf-9d52-534291b94287 None admin] RBAC: Authorizing `identity:get_project(project_id=d0157c96b92744a4b2956b1ef2c7d110)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.601643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec429f5d-5afb-4faf-9d52-534291b94287 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.603032 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 95/174] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/projects/d0157c96b92744a4b2956b1ef2c7d110 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:11.610135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec429f5d-5afb-4faf-9d52-534291b94287 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.619691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae5cc2-cd3e-4d4f-bf8e-6a0ca966bd04 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.676317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.676988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.677272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.678112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.678861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:11.679220 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.680639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.689774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:11.704139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 80/175] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:11.711059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.821308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec429f5d-5afb-4faf-9d52-534291b94287 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.867604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70550fc-5f45-46ba-aca5-0a8de10e82f0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.905913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c667da9-3b0a-47ed-ab4a-2902012cfddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.907606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c667da9-3b0a-47ed-ab4a-2902012cfddb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.908083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c667da9-3b0a-47ed-ab4a-2902012cfddb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.908546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c667da9-3b0a-47ed-ab4a-2902012cfddb None admin] PATH_INFO: `/v3/users/c956159832a6436bbf4c4a3517bd6170` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.908878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.909545 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.909831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.910111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.910777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c667da9-3b0a-47ed-ab4a-2902012cfddb None admin] RBAC: Authorizing `identity:get_user(user_id=c956159832a6436bbf4c4a3517bd6170)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.910949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] RBAC: Adding query filter params (role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=d0157c96b92744a4b2956b1ef2c7d110, user.id=c956159832a6436bbf4c4a3517bd6170) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:11.911465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.913101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c667da9-3b0a-47ed-ab4a-2902012cfddb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.913603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.915157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 81/176] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/users/c956159832a6436bbf4c4a3517bd6170 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:11.924438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c667da9-3b0a-47ed-ab4a-2902012cfddb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.929565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 96/177] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/role_assignments?user.id=c956159832a6436bbf4c4a3517bd6170&scope.project.id=d0157c96b92744a4b2956b1ef2c7d110&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:12.034243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:12.072260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c667da9-3b0a-47ed-ab4a-2902012cfddb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:12.144431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88656f4-172b-41a0-8196-3c7c10ccb9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:12.145460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88656f4-172b-41a0-8196-3c7c10ccb9e1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.145940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88656f4-172b-41a0-8196-3c7c10ccb9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.147001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88656f4-172b-41a0-8196-3c7c10ccb9e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.282744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88656f4-172b-41a0-8196-3c7c10ccb9e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:12.284603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88656f4-172b-41a0-8196-3c7c10ccb9e1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:12.417240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:12.491742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 82/178] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:12.500407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:12.541497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:12.567660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b3c9d-02dd-4c4a-bdbb-ce6c4decc492 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.567951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b3c9d-02dd-4c4a-bdbb-ce6c4decc492 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.568609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b3c9d-02dd-4c4a-bdbb-ce6c4decc492 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.570505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 83/179] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:12.580184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908c9d2-7687-403a-8838-5beafe7472c4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.580734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908c9d2-7687-403a-8838-5beafe7472c4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.581234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908c9d2-7687-403a-8838-5beafe7472c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.610573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a908c9d2-7687-403a-8838-5beafe7472c4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:12.612718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a908c9d2-7687-403a-8838-5beafe7472c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:12.644713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001cdd3-374b-4839-8b82-fa3004237376 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:12.712929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807bbd8-e28f-4598-8b7e-e7c6f50ddf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:12.713972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807bbd8-e28f-4598-8b7e-e7c6f50ddf73 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.714355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807bbd8-e28f-4598-8b7e-e7c6f50ddf73 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.714729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807bbd8-e28f-4598-8b7e-e7c6f50ddf73 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.720208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807bbd8-e28f-4598-8b7e-e7c6f50ddf73 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:12.720208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807bbd8-e28f-4598-8b7e-e7c6f50ddf73 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:12.720208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 97/180] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:12 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:12.727357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807bbd8-e28f-4598-8b7e-e7c6f50ddf73 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:12.737177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 84/181] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 15 15:32:12.773689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908c9d2-7687-403a-8838-5beafe7472c4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:12.845130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:12.852411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807bbd8-e28f-4598-8b7e-e7c6f50ddf73 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:12.877203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:12.884569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908c9d2-7687-403a-8838-5beafe7472c4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:12.891224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c52e3-1a6d-419a-8df6-842d19e8ddb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:12.891930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c52e3-1a6d-419a-8df6-842d19e8ddb8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.892221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c52e3-1a6d-419a-8df6-842d19e8ddb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.892820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c52e3-1a6d-419a-8df6-842d19e8ddb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.893333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c52e3-1a6d-419a-8df6-842d19e8ddb8 None admin] RBAC: Adding query filter params (role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=b7fddf3939de4bb99741de8e3307565f, user.id=c956159832a6436bbf4c4a3517bd6170) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:12.893644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c52e3-1a6d-419a-8df6-842d19e8ddb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:12.895105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c52e3-1a6d-419a-8df6-842d19e8ddb8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:12.906524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 98/182] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:12 2022] GET /identity/v3/role_assignments?user.id=c956159832a6436bbf4c4a3517bd6170&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:12.906524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad7a46-5daa-46df-9a98-5bf033689492 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.906524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad7a46-5daa-46df-9a98-5bf033689492 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.906524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad7a46-5daa-46df-9a98-5bf033689492 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.906524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 99/183] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:12.906524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21089152-6b6f-4a95-9970-1fa41533ac9c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.906986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21089152-6b6f-4a95-9970-1fa41533ac9c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.906986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21089152-6b6f-4a95-9970-1fa41533ac9c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.908371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 100/184] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:12.915324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4feb0-5608-4ba7-a96e-dac04504e759 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.915324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4feb0-5608-4ba7-a96e-dac04504e759 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.915324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4feb0-5608-4ba7-a96e-dac04504e759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.931407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c6c4feb0-5608-4ba7-a96e-dac04504e759 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:12.932731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c6c4feb0-5608-4ba7-a96e-dac04504e759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:12.933339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06357b5-a0c5-4e9f-89b6-e2bfe2b66887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:12.934165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06357b5-a0c5-4e9f-89b6-e2bfe2b66887 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.934447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06357b5-a0c5-4e9f-89b6-e2bfe2b66887 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.934721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06357b5-a0c5-4e9f-89b6-e2bfe2b66887 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:12.952463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06357b5-a0c5-4e9f-89b6-e2bfe2b66887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d06357b5-a0c5-4e9f-89b6-e2bfe2b66887 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:12.961387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:12.962350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:12.963181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 85/185] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:12.964814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff95e7-acff-49fc-8f0a-74419091edef None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:12.964814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff95e7-acff-49fc-8f0a-74419091edef None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:12.964814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff95e7-acff-49fc-8f0a-74419091edef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.016378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f3ff95e7-acff-49fc-8f0a-74419091edef None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:13.017633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f3ff95e7-acff-49fc-8f0a-74419091edef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:13.047385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 101/186] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:13.047385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d81d1-708e-4e32-8ed8-89233d5551f5 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.047385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d81d1-708e-4e32-8ed8-89233d5551f5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.047385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d81d1-708e-4e32-8ed8-89233d5551f5 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.047385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 102/187] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:13.051316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d81d1-708e-4e32-8ed8-89233d5551f5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.104807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 86/188] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:13.104807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1acb73-9db5-4f6f-8d50-3d04038566be None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.104807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1acb73-9db5-4f6f-8d50-3d04038566be None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.104807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1acb73-9db5-4f6f-8d50-3d04038566be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.122745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cd1acb73-9db5-4f6f-8d50-3d04038566be None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:13.124613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd1acb73-9db5-4f6f-8d50-3d04038566be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:13.157634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:13.170951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d81d1-708e-4e32-8ed8-89233d5551f5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.200047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 87/189] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:13.211493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1acb73-9db5-4f6f-8d50-3d04038566be None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.214592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.215204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.215534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.215810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.216554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] RBAC: Adding query filter params (name=reader) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:13.216855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:13.217420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:13.218605 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:13.227099 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 103/190] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:13.227942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.299926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1acb73-9db5-4f6f-8d50-3d04038566be None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.326505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315f9be-11d0-4388-a667-9f185e86378f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.332531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2e71b-582e-4315-aa28-c65f2c1f0d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.333335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2e71b-582e-4315-aa28-c65f2c1f0d16 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.333629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2e71b-582e-4315-aa28-c65f2c1f0d16 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.334412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2e71b-582e-4315-aa28-c65f2c1f0d16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.344745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2e71b-582e-4315-aa28-c65f2c1f0d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7fb2e71b-582e-4315-aa28-c65f2c1f0d16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:13.353203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:13.354068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:13.354846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 88/191] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:13.355524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2e71b-582e-4315-aa28-c65f2c1f0d16 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.360793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bc646-aab3-45c6-903e-f5a088d2d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.361590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc646-aab3-45c6-903e-f5a088d2d502 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.361877 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc646-aab3-45c6-903e-f5a088d2d502 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.362156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc646-aab3-45c6-903e-f5a088d2d502 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.363363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bc646-aab3-45c6-903e-f5a088d2d502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:13.364831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bc646-aab3-45c6-903e-f5a088d2d502 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:13.366501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 104/192] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:13.367254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bc646-aab3-45c6-903e-f5a088d2d502 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.391760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:13.431444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:13.459646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2e71b-582e-4315-aa28-c65f2c1f0d16 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.479308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bc646-aab3-45c6-903e-f5a088d2d502 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.492918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39081c71-782e-43fb-a47e-72d42092695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.493705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39081c71-782e-43fb-a47e-72d42092695a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.493986 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39081c71-782e-43fb-a47e-72d42092695a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.494259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39081c71-782e-43fb-a47e-72d42092695a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.504033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39081c71-782e-43fb-a47e-72d42092695a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-39081c71-782e-43fb-a47e-72d42092695a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:13.512068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:13.513100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:13.513879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:13.514457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a679a2-1991-4df4-ba9b-e1246c5f2102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.514864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 89/193] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:13.514864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39081c71-782e-43fb-a47e-72d42092695a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.515003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a679a2-1991-4df4-ba9b-e1246c5f2102 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.515003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a679a2-1991-4df4-ba9b-e1246c5f2102 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.515003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a679a2-1991-4df4-ba9b-e1246c5f2102 None admin] PATH_INFO: `/v3/users/4535c49340a94959bde8c258d072216d` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.519076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a679a2-1991-4df4-ba9b-e1246c5f2102 None admin] RBAC: Authorizing `identity:get_user(user_id=4535c49340a94959bde8c258d072216d)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:13.520595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a679a2-1991-4df4-ba9b-e1246c5f2102 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:13.522839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 105/194] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/users/4535c49340a94959bde8c258d072216d => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:13.523862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a679a2-1991-4df4-ba9b-e1246c5f2102 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.607998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:13.617214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39081c71-782e-43fb-a47e-72d42092695a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.622281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a679a2-1991-4df4-ba9b-e1246c5f2102 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.651558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.652219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.652504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.652781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.653525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:13.653839 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:13.655211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:13.657389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:13.658357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb23c-95e9-4ca7-9998-c3e570ecdb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.659043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb23c-95e9-4ca7-9998-c3e570ecdb88 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.659354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb23c-95e9-4ca7-9998-c3e570ecdb88 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.659636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbb23c-95e9-4ca7-9998-c3e570ecdb88 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.660347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb23c-95e9-4ca7-9998-c3e570ecdb88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:13.662305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbb23c-95e9-4ca7-9998-c3e570ecdb88 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:13.664411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 90/195] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:13.669238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.762660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee61b3-30a5-4d46-b6bf-65c96c8db9e3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.776365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 106/196] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:13 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:13.777238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb23c-95e9-4ca7-9998-c3e570ecdb88 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.812565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.813537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.813820 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.814264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.815563 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:13.815875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:13.817463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:13.832356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 91/197] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:13.832356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:13.895838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:13.901860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbb23c-95e9-4ca7-9998-c3e570ecdb88 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.940025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f263ac-e182-4890-a8d8-e7c0c732207c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:13.941246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e233c6-cac4-411b-9a82-9214f0510654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.941917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e233c6-cac4-411b-9a82-9214f0510654 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.942213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e233c6-cac4-411b-9a82-9214f0510654 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.942497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e233c6-cac4-411b-9a82-9214f0510654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:13.954946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:13.980334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefad0b6-5e71-4e62-adfc-c61600fccd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:13.982377 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefad0b6-5e71-4e62-adfc-c61600fccd88 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:13.982750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefad0b6-5e71-4e62-adfc-c61600fccd88 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:13.982942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefad0b6-5e71-4e62-adfc-c61600fccd88 None admin] PATH_INFO: `/v3/users/2c53ed74828b46b78bf15befd48ceda3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:14.000342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefad0b6-5e71-4e62-adfc-c61600fccd88 None admin] RBAC: Authorizing `identity:get_user(user_id=2c53ed74828b46b78bf15befd48ceda3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:14.002864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefad0b6-5e71-4e62-adfc-c61600fccd88 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:14.004686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 92/198] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/users/2c53ed74828b46b78bf15befd48ceda3 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:14.005461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefad0b6-5e71-4e62-adfc-c61600fccd88 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:14.041738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e233c6-cac4-411b-9a82-9214f0510654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:14.043173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e233c6-cac4-411b-9a82-9214f0510654 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:14.141485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefad0b6-5e71-4e62-adfc-c61600fccd88 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:14.164591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:14.171660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 107/199] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:14.173461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e233c6-cac4-411b-9a82-9214f0510654 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:14.186094 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279fd6c-e31a-495c-a15d-6c9184e2c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:14.186980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279fd6c-e31a-495c-a15d-6c9184e2c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:14.187266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279fd6c-e31a-495c-a15d-6c9184e2c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:14.187569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279fd6c-e31a-495c-a15d-6c9184e2c6c4 None admin] PATH_INFO: `/v3/users/2c53ed74828b46b78bf15befd48ceda3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:14.189989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279fd6c-e31a-495c-a15d-6c9184e2c6c4 None admin] RBAC: Authorizing `identity:get_user(user_id=2c53ed74828b46b78bf15befd48ceda3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:14.191495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279fd6c-e31a-495c-a15d-6c9184e2c6c4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:14.193849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 93/200] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:14 2022] GET /identity/v3/users/2c53ed74828b46b78bf15befd48ceda3 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:14.194467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279fd6c-e31a-495c-a15d-6c9184e2c6c4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:14.244675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:14.326532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e233c6-cac4-411b-9a82-9214f0510654 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:14.337181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279fd6c-e31a-495c-a15d-6c9184e2c6c4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:14.372719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524f3c5-6b3f-4309-ba81-ac38f4d809cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:14.373860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524f3c5-6b3f-4309-ba81-ac38f4d809cd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:14.373969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829a77-ee70-4d05-baed-face726b43f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:14.374486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524f3c5-6b3f-4309-ba81-ac38f4d809cd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:14.374801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829a77-ee70-4d05-baed-face726b43f7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:14.375083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829a77-ee70-4d05-baed-face726b43f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:14.375397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829a77-ee70-4d05-baed-face726b43f7 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:14.375660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524f3c5-6b3f-4309-ba81-ac38f4d809cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:14.376600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82829a77-ee70-4d05-baed-face726b43f7 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:14.378136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82829a77-ee70-4d05-baed-face726b43f7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:14.385528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 94/201] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:14 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:14.386438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829a77-ee70-4d05-baed-face726b43f7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:14.451045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:14.469157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524f3c5-6b3f-4309-ba81-ac38f4d809cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:14.475163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524f3c5-6b3f-4309-ba81-ac38f4d809cd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:14.558058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:14.565374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829a77-ee70-4d05-baed-face726b43f7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:14.642753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6904fa-6d29-4b9e-9cce-e0d85131d574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:14.643436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6904fa-6d29-4b9e-9cce-e0d85131d574 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:14.643722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6904fa-6d29-4b9e-9cce-e0d85131d574 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:14.643998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6904fa-6d29-4b9e-9cce-e0d85131d574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:14.659053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 108/202] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:14.659912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524f3c5-6b3f-4309-ba81-ac38f4d809cd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:14.770508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6904fa-6d29-4b9e-9cce-e0d85131d574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:14.774056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6904fa-6d29-4b9e-9cce-e0d85131d574 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:14.846523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524f3c5-6b3f-4309-ba81-ac38f4d809cd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:14.883678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:14.903883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:14.904950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:14.907890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:14.908951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:14.910204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] RBAC: Adding query filter params (role.id=3555f68f0fca46018de9022f06dcdc90, scope.project.id=b7fddf3939de4bb99741de8e3307565f, user.id=4535c49340a94959bde8c258d072216d) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:14.910721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:14.913889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:14.922297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 109/203] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:14 2022] GET /identity/v3/role_assignments?user.id=4535c49340a94959bde8c258d072216d&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=3555f68f0fca46018de9022f06dcdc90 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:14.930446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:14.956162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:14.983977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 95/204] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:14.989364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6904fa-6d29-4b9e-9cce-e0d85131d574 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:15.136912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932207b-100c-49c4-afb6-c5710a1d2e43 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:15.202866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3411e-52c2-460a-8270-26b67eaead6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:15.202866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3411e-52c2-460a-8270-26b67eaead6c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:15.203106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3411e-52c2-460a-8270-26b67eaead6c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:15.203408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3411e-52c2-460a-8270-26b67eaead6c None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:15.205686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3411e-52c2-460a-8270-26b67eaead6c None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:15.207282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3411e-52c2-460a-8270-26b67eaead6c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:15.209313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 110/205] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:14 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:15.216391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3411e-52c2-460a-8270-26b67eaead6c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:15.265594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6904fa-6d29-4b9e-9cce-e0d85131d574 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:15.336419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:15.350855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be08aa-764a-43cb-8d2c-f70929e05b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:15.352226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be08aa-764a-43cb-8d2c-f70929e05b4a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:15.352707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be08aa-764a-43cb-8d2c-f70929e05b4a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:15.353212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be08aa-764a-43cb-8d2c-f70929e05b4a None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:15.355415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be08aa-764a-43cb-8d2c-f70929e05b4a None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:15.359518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be08aa-764a-43cb-8d2c-f70929e05b4a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:15.362605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 96/206] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:14 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:15.370654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be08aa-764a-43cb-8d2c-f70929e05b4a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:15.395187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3411e-52c2-460a-8270-26b67eaead6c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:15.437775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:15.442094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410d7a2-4593-442c-983c-79c3839c9f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:15.443758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d7a2-4593-442c-983c-79c3839c9f16 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:15.444050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d7a2-4593-442c-983c-79c3839c9f16 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:15.444338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410d7a2-4593-442c-983c-79c3839c9f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:15.446411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410d7a2-4593-442c-983c-79c3839c9f16 None admin] RBAC: Adding query filter params (role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=366bead620044a59b9ae769237db8568, user.id=2c53ed74828b46b78bf15befd48ceda3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:15.446411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410d7a2-4593-442c-983c-79c3839c9f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:15.447685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410d7a2-4593-442c-983c-79c3839c9f16 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:15.459798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 111/207] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:15 2022] GET /identity/v3/role_assignments?user.id=2c53ed74828b46b78bf15befd48ceda3&scope.project.id=366bead620044a59b9ae769237db8568&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:15.536720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be08aa-764a-43cb-8d2c-f70929e05b4a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:15.578148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23af4f8-3ae1-483b-9762-6754d811c85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:15.579184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23af4f8-3ae1-483b-9762-6754d811c85e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:15.579702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23af4f8-3ae1-483b-9762-6754d811c85e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:15.580329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23af4f8-3ae1-483b-9762-6754d811c85e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:15.581482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23af4f8-3ae1-483b-9762-6754d811c85e None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=366bead620044a59b9ae769237db8568, user.id=2c53ed74828b46b78bf15befd48ceda3) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:15.582106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23af4f8-3ae1-483b-9762-6754d811c85e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:15.585178 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23af4f8-3ae1-483b-9762-6754d811c85e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:15.595366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 97/208] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:15 2022] GET /identity/v3/role_assignments?user.id=2c53ed74828b46b78bf15befd48ceda3&scope.project.id=366bead620044a59b9ae769237db8568&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:16.124897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11825ab6-9a7c-44e3-98fb-a80fb97b4e15 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.125398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11825ab6-9a7c-44e3-98fb-a80fb97b4e15 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.125499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11825ab6-9a7c-44e3-98fb-a80fb97b4e15 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.127142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 112/209] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:16.137822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d449c-09d9-4432-b49b-f105a8f5e08d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.138265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d449c-09d9-4432-b49b-f105a8f5e08d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.138445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d449c-09d9-4432-b49b-f105a8f5e08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.179881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-340d449c-09d9-4432-b49b-f105a8f5e08d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:16.180640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a20c-d457-4e79-b9f4-d50746612095 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.180935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a20c-d457-4e79-b9f4-d50746612095 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.181141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-340d449c-09d9-4432-b49b-f105a8f5e08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:16.181239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a20c-d457-4e79-b9f4-d50746612095 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.182746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 113/210] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:16.187759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799745-de99-4602-9fcc-6d7a80121607 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.188054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799745-de99-4602-9fcc-6d7a80121607 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.188338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc799745-de99-4602-9fcc-6d7a80121607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.206792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dc799745-de99-4602-9fcc-6d7a80121607 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:16.208051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dc799745-de99-4602-9fcc-6d7a80121607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:16.290225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 98/211] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:16.315516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d449c-09d9-4432-b49b-f105a8f5e08d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:16.338507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:16.345015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 114/212] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:16.376469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc799745-de99-4602-9fcc-6d7a80121607 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:16.446281 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d449c-09d9-4432-b49b-f105a8f5e08d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:16.482258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:16.492527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42718d5f-a7ea-4be5-8539-56868fbf2b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:16.493331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42718d5f-a7ea-4be5-8539-56868fbf2b86 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.493626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42718d5f-a7ea-4be5-8539-56868fbf2b86 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.502836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42718d5f-a7ea-4be5-8539-56868fbf2b86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.513811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42718d5f-a7ea-4be5-8539-56868fbf2b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-42718d5f-a7ea-4be5-8539-56868fbf2b86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:16.522938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:16.524449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:16.525222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:16.527808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 99/213] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:16.536620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc799745-de99-4602-9fcc-6d7a80121607 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:16.546983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42718d5f-a7ea-4be5-8539-56868fbf2b86 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:16.566060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9cc03-766a-4b04-9032-cad64695ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:16.566867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9cc03-766a-4b04-9032-cad64695ab8c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.567521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9cc03-766a-4b04-9032-cad64695ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.567651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9cc03-766a-4b04-9032-cad64695ab8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.571028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9cc03-766a-4b04-9032-cad64695ab8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:16.571028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9cc03-766a-4b04-9032-cad64695ab8c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:16.579684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 115/214] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:16.580733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9cc03-766a-4b04-9032-cad64695ab8c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:16.668767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:16.695342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42718d5f-a7ea-4be5-8539-56868fbf2b86 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:16.717570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9cc03-766a-4b04-9032-cad64695ab8c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:16.743657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:16.745381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.745670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.745948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.746834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:16.747150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:16.748614 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:16.760031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 100/215] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:16.767030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:16.787316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098172-151a-4905-9d2b-c74de729377d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:16.787316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098172-151a-4905-9d2b-c74de729377d None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.787316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098172-151a-4905-9d2b-c74de729377d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.787316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098172-151a-4905-9d2b-c74de729377d None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.787712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3098172-151a-4905-9d2b-c74de729377d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:16.790472 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3098172-151a-4905-9d2b-c74de729377d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:16.807722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:16.911065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9d09a-b9f0-45d4-b84a-7ce24204652a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:16.943988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 116/216] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:16 2022] POST /identity/v3/users => generated 297 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:16.964484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2843418-5814-4a54-9d50-d487c29868dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:16.965630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2843418-5814-4a54-9d50-d487c29868dc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:16.966106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2843418-5814-4a54-9d50-d487c29868dc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:16.966562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2843418-5814-4a54-9d50-d487c29868dc None admin] PATH_INFO: `/v3/users/3c9b6834a7774691aa6b08b7a53ea7db` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:16.976402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:16.984812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2843418-5814-4a54-9d50-d487c29868dc None admin] RBAC: Authorizing `identity:get_user(user_id=3c9b6834a7774691aa6b08b7a53ea7db)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:16.986733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2843418-5814-4a54-9d50-d487c29868dc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:16.994335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 101/217] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:16 2022] GET /identity/v3/users/3c9b6834a7774691aa6b08b7a53ea7db => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:16.996221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098172-151a-4905-9d2b-c74de729377d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:17.121923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098172-151a-4905-9d2b-c74de729377d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:17.164747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4578fed-8e0a-4100-92a1-35d8f747eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:17.166078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578fed-8e0a-4100-92a1-35d8f747eb11 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:17.166492 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578fed-8e0a-4100-92a1-35d8f747eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:17.166891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4578fed-8e0a-4100-92a1-35d8f747eb11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:17.231510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:17.243758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4578fed-8e0a-4100-92a1-35d8f747eb11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:17.244944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4578fed-8e0a-4100-92a1-35d8f747eb11 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:17.365731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 117/218] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:17.374017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2843418-5814-4a54-9d50-d487c29868dc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:17.505352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2843418-5814-4a54-9d50-d487c29868dc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:17.542303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dfe5b-4ed5-4dcb-a810-b62246771047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:17.543488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dfe5b-4ed5-4dcb-a810-b62246771047 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:17.543918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dfe5b-4ed5-4dcb-a810-b62246771047 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:17.544336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dfe5b-4ed5-4dcb-a810-b62246771047 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:17.549579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dfe5b-4ed5-4dcb-a810-b62246771047 None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:17.551263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dfe5b-4ed5-4dcb-a810-b62246771047 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:17.553212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 102/219] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:17 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:17.560843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4578fed-8e0a-4100-92a1-35d8f747eb11 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:17.723657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:17.729696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4578fed-8e0a-4100-92a1-35d8f747eb11 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:17.787959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e967bff-dfae-41a2-8036-baeba5e1fa49 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:17.787959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e967bff-dfae-41a2-8036-baeba5e1fa49 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:17.787959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e967bff-dfae-41a2-8036-baeba5e1fa49 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:17.788717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 103/220] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:17.795607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a3c76-859c-4a5d-94a3-e085c25a9c43 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:17.795901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a3c76-859c-4a5d-94a3-e085c25a9c43 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:17.796187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a3c76-859c-4a5d-94a3-e085c25a9c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:17.798204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff4f28-7e40-4f11-8113-a1d475417cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:17.798877 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff4f28-7e40-4f11-8113-a1d475417cb4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:17.799160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff4f28-7e40-4f11-8113-a1d475417cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:17.799511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff4f28-7e40-4f11-8113-a1d475417cb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:17.800362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff4f28-7e40-4f11-8113-a1d475417cb4 None admin] RBAC: Adding query filter params (role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=366bead620044a59b9ae769237db8568, user.id=3c9b6834a7774691aa6b08b7a53ea7db) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:17.800677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff4f28-7e40-4f11-8113-a1d475417cb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:17.802155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff4f28-7e40-4f11-8113-a1d475417cb4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:17.823668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3c5a3c76-859c-4a5d-94a3-e085c25a9c43 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:17.825210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3c5a3c76-859c-4a5d-94a3-e085c25a9c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:17.834754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 118/221] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:17 2022] GET /identity/v3/role_assignments?user.id=3c9b6834a7774691aa6b08b7a53ea7db&scope.project.id=366bead620044a59b9ae769237db8568&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:17.942471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2ec1-9d20-41b0-9574-e7025416e901 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:17.942760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2ec1-9d20-41b0-9574-e7025416e901 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:17.943629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2ec1-9d20-41b0-9574-e7025416e901 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:17.945124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 119/222] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:17.954288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b4e7-a835-48a7-b977-07f8560decce None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:17.954571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b4e7-a835-48a7-b977-07f8560decce None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:17.954859 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b4e7-a835-48a7-b977-07f8560decce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:17.984265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:17.985231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-62b8b4e7-a835-48a7-b977-07f8560decce None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:17.986503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-62b8b4e7-a835-48a7-b977-07f8560decce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:18.004093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 104/223] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:18.058166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a3c76-859c-4a5d-94a3-e085c25a9c43 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:18.142145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 120/224] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:18.207319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8b4e7-a835-48a7-b977-07f8560decce None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:18.262829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a3c76-859c-4a5d-94a3-e085c25a9c43 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:18.353976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41809ebd-7195-4f79-a078-8c7cd6f1a2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:18.354706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41809ebd-7195-4f79-a078-8c7cd6f1a2f8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:18.354988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41809ebd-7195-4f79-a078-8c7cd6f1a2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:18.355309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41809ebd-7195-4f79-a078-8c7cd6f1a2f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:18.378368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41809ebd-7195-4f79-a078-8c7cd6f1a2f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:18.380617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-41809ebd-7195-4f79-a078-8c7cd6f1a2f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:18.386641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:18.391065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:18.391933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:18.395958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 105/225] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:18.404091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41809ebd-7195-4f79-a078-8c7cd6f1a2f8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:18.414052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8b4e7-a835-48a7-b977-07f8560decce None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:18.492681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1d89c-cc5a-4265-b667-6f7d356ad3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:18.493524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d89c-cc5a-4265-b667-6f7d356ad3f3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:18.494774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d89c-cc5a-4265-b667-6f7d356ad3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:18.494774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1d89c-cc5a-4265-b667-6f7d356ad3f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:18.514420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:18.527053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1d89c-cc5a-4265-b667-6f7d356ad3f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-55a1d89c-cc5a-4265-b667-6f7d356ad3f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:18.537818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:18.540232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:18.540992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 121/226] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:32:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:18.555576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1d89c-cc5a-4265-b667-6f7d356ad3f3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:18.603060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41809ebd-7195-4f79-a078-8c7cd6f1a2f8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:18.695405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:18.695405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:18.695405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:18.695405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:18.695857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:18.696173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:18.697622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:18.723896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 106/227] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:18.729434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:18.784372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1d89c-cc5a-4265-b667-6f7d356ad3f3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:18.860818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:18.861676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:18.861964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:18.862242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:18.862998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:18.863383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:18.865924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:18.878744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 122/228] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:18.884952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:18.911646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:18.970251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ca86e-cc06-4833-84f7-a7984f3aa3f8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:19.004280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:19.046744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fd34f-56aa-4819-97d5-652ea7523738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:19.047680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fd34f-56aa-4819-97d5-652ea7523738 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.047966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fd34f-56aa-4819-97d5-652ea7523738 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.048242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fd34f-56aa-4819-97d5-652ea7523738 None admin] PATH_INFO: `/v3/users/2c53ed74828b46b78bf15befd48ceda3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.053437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fd34f-56aa-4819-97d5-652ea7523738 None admin] RBAC: Authorizing `identity:get_user(user_id=2c53ed74828b46b78bf15befd48ceda3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:19.054922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fd34f-56aa-4819-97d5-652ea7523738 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:19.058483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 107/229] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:18 2022] GET /identity/v3/users/2c53ed74828b46b78bf15befd48ceda3 => generated 283 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:19.063723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fd34f-56aa-4819-97d5-652ea7523738 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:19.103102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478c57c-7518-4084-9ad5-ee13baa216c5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:19.176652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5340a8-bed6-4801-baae-948ea970b60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:19.178153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5340a8-bed6-4801-baae-948ea970b60d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.178437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5340a8-bed6-4801-baae-948ea970b60d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.178712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5340a8-bed6-4801-baae-948ea970b60d None admin] PATH_INFO: `/v3/users/2c53ed74828b46b78bf15befd48ceda3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.183102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5340a8-bed6-4801-baae-948ea970b60d None admin] RBAC: Authorizing `identity:get_user(user_id=2c53ed74828b46b78bf15befd48ceda3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:19.184645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5340a8-bed6-4801-baae-948ea970b60d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:19.186343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 123/230] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:18 2022] GET /identity/v3/users/2c53ed74828b46b78bf15befd48ceda3 => generated 283 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:19.191931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5340a8-bed6-4801-baae-948ea970b60d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:19.269238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fd34f-56aa-4819-97d5-652ea7523738 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:19.329174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c25a4-e2b1-4377-b6b6-be331e2ecc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:19.330552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c25a4-e2b1-4377-b6b6-be331e2ecc0f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.331026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c25a4-e2b1-4377-b6b6-be331e2ecc0f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.331422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c25a4-e2b1-4377-b6b6-be331e2ecc0f None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.335856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c25a4-e2b1-4377-b6b6-be331e2ecc0f None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:19.337474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c25a4-e2b1-4377-b6b6-be331e2ecc0f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:19.341999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 108/231] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:19 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:19.348715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c25a4-e2b1-4377-b6b6-be331e2ecc0f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:19.445328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:19.457477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:19.516315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c25a4-e2b1-4377-b6b6-be331e2ecc0f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:19.545442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5340a8-bed6-4801-baae-948ea970b60d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:19.568052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211230a-b6bf-43f3-afe1-ecb517e081fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:19.569128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211230a-b6bf-43f3-afe1-ecb517e081fc None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.569490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211230a-b6bf-43f3-afe1-ecb517e081fc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.569845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211230a-b6bf-43f3-afe1-ecb517e081fc None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568/users/2c53ed74828b46b78bf15befd48ceda3/roles/9a22f6f23fb64da488d982c17ed565ce` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.585617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211230a-b6bf-43f3-afe1-ecb517e081fc None admin] RBAC: Authorizing `identity:create_grant(project_id=366bead620044a59b9ae769237db8568, user_id=2c53ed74828b46b78bf15befd48ceda3, role_id=9a22f6f23fb64da488d982c17ed565ce)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:19.586711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211230a-b6bf-43f3-afe1-ecb517e081fc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:19.604420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276ccc5-fc0b-4bde-b71f-536ab753d09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:19.605792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276ccc5-fc0b-4bde-b71f-536ab753d09b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.606292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276ccc5-fc0b-4bde-b71f-536ab753d09b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.606776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276ccc5-fc0b-4bde-b71f-536ab753d09b None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.608349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276ccc5-fc0b-4bde-b71f-536ab753d09b None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:19.610156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276ccc5-fc0b-4bde-b71f-536ab753d09b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:19.612610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 124/232] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:19 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:19.619258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276ccc5-fc0b-4bde-b71f-536ab753d09b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:19.709470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 109/233] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:19 2022] PUT /identity/v3/projects/366bead620044a59b9ae769237db8568/users/2c53ed74828b46b78bf15befd48ceda3/roles/9a22f6f23fb64da488d982c17ed565ce => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:19.740554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276ccc5-fc0b-4bde-b71f-536ab753d09b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:19.780447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37efa4b-41c2-4ccb-a0ea-dff57ac97958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:19.781741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37efa4b-41c2-4ccb-a0ea-dff57ac97958 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.782237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37efa4b-41c2-4ccb-a0ea-dff57ac97958 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.782736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37efa4b-41c2-4ccb-a0ea-dff57ac97958 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568/users/2c53ed74828b46b78bf15befd48ceda3/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.785159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37efa4b-41c2-4ccb-a0ea-dff57ac97958 None admin] RBAC: Authorizing `identity:create_grant(project_id=366bead620044a59b9ae769237db8568, user_id=2c53ed74828b46b78bf15befd48ceda3, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:19.787553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37efa4b-41c2-4ccb-a0ea-dff57ac97958 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:19.825605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31125b1-19db-4c7d-80cf-860c71f2f3b1 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.825883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31125b1-19db-4c7d-80cf-860c71f2f3b1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.826101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31125b1-19db-4c7d-80cf-860c71f2f3b1 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.827786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 110/234] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:19.835332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032134-92fa-42a4-918d-374747ff912f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.835332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032134-92fa-42a4-918d-374747ff912f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.835332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032134-92fa-42a4-918d-374747ff912f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.856400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ee032134-92fa-42a4-918d-374747ff912f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:19.857684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ee032134-92fa-42a4-918d-374747ff912f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:19.905624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 125/235] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:19 2022] PUT /identity/v3/projects/366bead620044a59b9ae769237db8568/users/2c53ed74828b46b78bf15befd48ceda3/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:19.911016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915431b2-e61d-41f0-aab2-6264e42178c2 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.911016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915431b2-e61d-41f0-aab2-6264e42178c2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.911016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915431b2-e61d-41f0-aab2-6264e42178c2 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.911016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 126/236] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:19.921280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8d2e9-c5e0-41a9-8c0d-fe7d10c29e6c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:19.924883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8d2e9-c5e0-41a9-8c0d-fe7d10c29e6c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:19.924883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8d2e9-c5e0-41a9-8c0d-fe7d10c29e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:19.936745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:19.955334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6ba8d2e9-c5e0-41a9-8c0d-fe7d10c29e6c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:19.957286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6ba8d2e9-c5e0-41a9-8c0d-fe7d10c29e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:19.969656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 111/237] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:19.997123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032134-92fa-42a4-918d-374747ff912f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:20.052063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:20.069821 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 127/238] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:20.097984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8d2e9-c5e0-41a9-8c0d-fe7d10c29e6c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:20.157032 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032134-92fa-42a4-918d-374747ff912f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:20.217955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da98fe7-9e9b-49e8-a5b9-0dd16f104225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:20.219304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98fe7-9e9b-49e8-a5b9-0dd16f104225 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:20.219704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98fe7-9e9b-49e8-a5b9-0dd16f104225 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:20.219990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98fe7-9e9b-49e8-a5b9-0dd16f104225 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:20.236788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da98fe7-9e9b-49e8-a5b9-0dd16f104225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3da98fe7-9e9b-49e8-a5b9-0dd16f104225 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:20.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:20.248736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:20.249490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:20.250064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8d2e9-c5e0-41a9-8c0d-fe7d10c29e6c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:20.250142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 112/239] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:20.259318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da98fe7-9e9b-49e8-a5b9-0dd16f104225 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:20.323409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88763f59-a382-44b4-a2ac-8786aef7b9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:20.324177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88763f59-a382-44b4-a2ac-8786aef7b9cc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:20.324464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88763f59-a382-44b4-a2ac-8786aef7b9cc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:20.324743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88763f59-a382-44b4-a2ac-8786aef7b9cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:20.326078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88763f59-a382-44b4-a2ac-8786aef7b9cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:20.327576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88763f59-a382-44b4-a2ac-8786aef7b9cc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:20.332699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 128/240] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:20.338765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88763f59-a382-44b4-a2ac-8786aef7b9cc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:20.344661 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:20.466577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da98fe7-9e9b-49e8-a5b9-0dd16f104225 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:20.471603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:20.510726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de22773-cae8-417a-8bd8-a1f4568af022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:20.513311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22773-cae8-417a-8bd8-a1f4568af022 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:20.513612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22773-cae8-417a-8bd8-a1f4568af022 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:20.513899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22773-cae8-417a-8bd8-a1f4568af022 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:20.514819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de22773-cae8-417a-8bd8-a1f4568af022 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:20.515134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de22773-cae8-417a-8bd8-a1f4568af022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:20.519375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de22773-cae8-417a-8bd8-a1f4568af022 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:20.536363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 113/241] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:20.537545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98f67-c305-48cf-b6cb-9f715c1aad29 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:20.537834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98f67-c305-48cf-b6cb-9f715c1aad29 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:20.538111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c98f67-c305-48cf-b6cb-9f715c1aad29 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:20.539401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 114/242] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:20.545822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c98f67-c305-48cf-b6cb-9f715c1aad29 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:20.546867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88763f59-a382-44b4-a2ac-8786aef7b9cc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:20.616970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483d5ee-9ca3-44a2-a3b4-6a134749f0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:20.617807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483d5ee-9ca3-44a2-a3b4-6a134749f0e4 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:20.618229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483d5ee-9ca3-44a2-a3b4-6a134749f0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:20.618520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483d5ee-9ca3-44a2-a3b4-6a134749f0e4 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:20.619248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483d5ee-9ca3-44a2-a3b4-6a134749f0e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:20.624513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483d5ee-9ca3-44a2-a3b4-6a134749f0e4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:20.669849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c98f67-c305-48cf-b6cb-9f715c1aad29 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:20.709423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933bcc7-80f6-4d23-8852-02526aef0ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:20.710612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933bcc7-80f6-4d23-8852-02526aef0ec7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:20.710964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933bcc7-80f6-4d23-8852-02526aef0ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:20.711245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933bcc7-80f6-4d23-8852-02526aef0ec7 None admin] PATH_INFO: `/v3/users/a2901d3ff8184dff9e6c0a54146f6809` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:20.726213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933bcc7-80f6-4d23-8852-02526aef0ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=a2901d3ff8184dff9e6c0a54146f6809)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:20.727853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933bcc7-80f6-4d23-8852-02526aef0ec7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:20.729742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 115/243] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:20 2022] GET /identity/v3/users/a2901d3ff8184dff9e6c0a54146f6809 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:20.731263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4666-be06-4548-864c-631157cf8b37 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:20.731559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4666-be06-4548-864c-631157cf8b37 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:20.731840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4666-be06-4548-864c-631157cf8b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:20.751007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bf9c4666-be06-4548-864c-631157cf8b37 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:20.752127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bf9c4666-be06-4548-864c-631157cf8b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:20.756641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 129/244] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:20 2022] POST /identity/v3/users => generated 293 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:20.757709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483d5ee-9ca3-44a2-a3b4-6a134749f0e4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:20.772498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:20.878507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 116/245] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:20.911685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:20.916429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4666-be06-4548-864c-631157cf8b37 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:20.993893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483d5ee-9ca3-44a2-a3b4-6a134749f0e4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:21.057781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce08c78-bd01-4341-853c-7f6fe01d4de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:21.058460 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce08c78-bd01-4341-853c-7f6fe01d4de2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:21.058884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce08c78-bd01-4341-853c-7f6fe01d4de2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:21.059159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce08c78-bd01-4341-853c-7f6fe01d4de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:21.077136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4666-be06-4548-864c-631157cf8b37 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:21.117931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416bd7a-2685-4e98-8e70-3ff6377b73ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:21.118800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416bd7a-2685-4e98-8e70-3ff6377b73ff None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:21.119089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416bd7a-2685-4e98-8e70-3ff6377b73ff None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:21.119399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416bd7a-2685-4e98-8e70-3ff6377b73ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:21.139318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416bd7a-2685-4e98-8e70-3ff6377b73ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a416bd7a-2685-4e98-8e70-3ff6377b73ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:21.145222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:21.146069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:21.147222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:21.148035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce08c78-bd01-4341-853c-7f6fe01d4de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:21.148118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 117/246] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:21.148448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce08c78-bd01-4341-853c-7f6fe01d4de2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:21.153661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416bd7a-2685-4e98-8e70-3ff6377b73ff None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:21.173555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:21.228171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:21.292761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 130/247] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:21.295909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce08c78-bd01-4341-853c-7f6fe01d4de2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:21.296932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416bd7a-2685-4e98-8e70-3ff6377b73ff None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:21.341971 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:21.343125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:21.343751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:21.344229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:21.345178 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:21.345684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:21.347323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:21.358783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 118/248] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:21.364755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:21.433352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce08c78-bd01-4341-853c-7f6fe01d4de2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:21.470675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0bcfe-d230-47a3-8a63-ebb9f3cb41af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:21.472837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0bcfe-d230-47a3-8a63-ebb9f3cb41af None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:21.472837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0bcfe-d230-47a3-8a63-ebb9f3cb41af None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:21.472837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0bcfe-d230-47a3-8a63-ebb9f3cb41af None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:21.473756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0bcfe-d230-47a3-8a63-ebb9f3cb41af None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:21.475572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0bcfe-d230-47a3-8a63-ebb9f3cb41af None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:21.477813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 131/249] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:21 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:21.479476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:21.483335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0bcfe-d230-47a3-8a63-ebb9f3cb41af None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:21.512692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c808cf2-b3b3-429f-92d0-6c707ab07823 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:21.520905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:21.605965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad68a9-1f5b-4227-8363-02644fdf4c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:21.606848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad68a9-1f5b-4227-8363-02644fdf4c1b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:21.607135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad68a9-1f5b-4227-8363-02644fdf4c1b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:21.607437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad68a9-1f5b-4227-8363-02644fdf4c1b None admin] PATH_INFO: `/v3/users/3c9b6834a7774691aa6b08b7a53ea7db` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:21.608655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad68a9-1f5b-4227-8363-02644fdf4c1b None admin] RBAC: Authorizing `identity:get_user(user_id=3c9b6834a7774691aa6b08b7a53ea7db)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:21.610131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad68a9-1f5b-4227-8363-02644fdf4c1b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:21.616952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 119/250] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:21 2022] GET /identity/v3/users/3c9b6834a7774691aa6b08b7a53ea7db => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:21.617804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad68a9-1f5b-4227-8363-02644fdf4c1b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:21.669744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0bcfe-d230-47a3-8a63-ebb9f3cb41af None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:21.774186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:21.775003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:21.775313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:21.775598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:21.776449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] RBAC: Adding query filter params (role.id=3555f68f0fca46018de9022f06dcdc90, scope.project.id=366bead620044a59b9ae769237db8568, user.id=a2901d3ff8184dff9e6c0a54146f6809) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:21.776765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:21.778504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:21.795969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 132/251] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:21 2022] GET /identity/v3/role_assignments?user.id=a2901d3ff8184dff9e6c0a54146f6809&scope.project.id=366bead620044a59b9ae769237db8568&role.id=3555f68f0fca46018de9022f06dcdc90 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:21.858200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad68a9-1f5b-4227-8363-02644fdf4c1b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:21.948187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf539f9-5271-4825-8025-b558518dcaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:21.949482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf539f9-5271-4825-8025-b558518dcaa3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:21.949971 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf539f9-5271-4825-8025-b558518dcaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:21.950446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf539f9-5271-4825-8025-b558518dcaa3 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:21.952122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf539f9-5271-4825-8025-b558518dcaa3 None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:21.953909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf539f9-5271-4825-8025-b558518dcaa3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:21.956057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 120/252] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:21 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:21.963504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:22.148423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:22.163530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380ddda-d756-49fd-afcf-df63d7344919 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:22.216774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e4a76-292d-4b8a-b651-69ac1599f729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:22.218032 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e4a76-292d-4b8a-b651-69ac1599f729 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:22.218549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e4a76-292d-4b8a-b651-69ac1599f729 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:22.219043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e4a76-292d-4b8a-b651-69ac1599f729 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568/users/3c9b6834a7774691aa6b08b7a53ea7db/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:22.225189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e4a76-292d-4b8a-b651-69ac1599f729 None admin] RBAC: Authorizing `identity:create_grant(project_id=366bead620044a59b9ae769237db8568, user_id=3c9b6834a7774691aa6b08b7a53ea7db, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:22.226921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e4a76-292d-4b8a-b651-69ac1599f729 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:22.297701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 133/253] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:21 2022] PUT /identity/v3/projects/366bead620044a59b9ae769237db8568/users/3c9b6834a7774691aa6b08b7a53ea7db/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:22.633933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3147c4a-ead9-4ac2-b34e-d490efd0856a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:22.633933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3147c4a-ead9-4ac2-b34e-d490efd0856a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:22.633933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3147c4a-ead9-4ac2-b34e-d490efd0856a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:22.634916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 121/254] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:22.641971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b2eb5-d612-4d15-88d7-fd554fadcb72 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:22.641971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b2eb5-d612-4d15-88d7-fd554fadcb72 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:22.641971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b2eb5-d612-4d15-88d7-fd554fadcb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:22.664389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ce5b2eb5-d612-4d15-88d7-fd554fadcb72 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:22.665907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ce5b2eb5-d612-4d15-88d7-fd554fadcb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:22.730628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890c7e6-c04e-4aeb-b125-eda7da2e6df6 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:22.731441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890c7e6-c04e-4aeb-b125-eda7da2e6df6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:22.731933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890c7e6-c04e-4aeb-b125-eda7da2e6df6 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:22.733515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 122/255] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:22.743330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5bf0d-e6f3-4491-abfa-101efefec41f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:22.743330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5bf0d-e6f3-4491-abfa-101efefec41f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:22.743330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5bf0d-e6f3-4491-abfa-101efefec41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:22.763104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 134/256] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:22.788508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e6d5bf0d-e6f3-4491-abfa-101efefec41f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:22.789710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e6d5bf0d-e6f3-4491-abfa-101efefec41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:22.795370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:22.842754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b2eb5-d612-4d15-88d7-fd554fadcb72 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:22.983403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 123/257] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:23.004034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:23.023448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5bf0d-e6f3-4491-abfa-101efefec41f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:23.030965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b2eb5-d612-4d15-88d7-fd554fadcb72 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:23.100123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49635c67-e400-4c43-9e6a-9ef609615286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:23.101322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49635c67-e400-4c43-9e6a-9ef609615286 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:23.101819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49635c67-e400-4c43-9e6a-9ef609615286 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:23.102302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49635c67-e400-4c43-9e6a-9ef609615286 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:23.121172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49635c67-e400-4c43-9e6a-9ef609615286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-49635c67-e400-4c43-9e6a-9ef609615286 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:23.149532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:23.150457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:23.151255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 135/258] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:32:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:23.157040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49635c67-e400-4c43-9e6a-9ef609615286 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:23.360265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5bf0d-e6f3-4491-abfa-101efefec41f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:23.456986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49635c67-e400-4c43-9e6a-9ef609615286 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:23.462507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9493f-5f53-4f8e-b65a-07f1360d3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:23.463358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9493f-5f53-4f8e-b65a-07f1360d3759 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:23.463653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9493f-5f53-4f8e-b65a-07f1360d3759 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:23.463936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9493f-5f53-4f8e-b65a-07f1360d3759 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:23.495718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:23.504173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9493f-5f53-4f8e-b65a-07f1360d3759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-cbf9493f-5f53-4f8e-b65a-07f1360d3759 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:23.514067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:23.515035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:23.515871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:23.524000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 124/259] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:23.530755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9493f-5f53-4f8e-b65a-07f1360d3759 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:23.557369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:23.558550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:23.559037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:23.559548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:23.560642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:23.561148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:23.562872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:23.591022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 136/260] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:23.608953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:23.621589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:23.837203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9493f-5f53-4f8e-b65a-07f1360d3759 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:23.879072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc5a32-1b14-4484-83b3-7439f93dde53 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:23.921462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:23.922156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:23.922446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:23.922728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:23.923551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:23.923872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:23.925287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:23.946463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 125/261] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:23.952557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:23.996021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d9f24-821a-4058-ad8b-01bb160e2410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:23.997322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9f24-821a-4058-ad8b-01bb160e2410 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:23.997808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9f24-821a-4058-ad8b-01bb160e2410 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:23.998419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9f24-821a-4058-ad8b-01bb160e2410 None admin] PATH_INFO: `/v3/users/2c53ed74828b46b78bf15befd48ceda3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.000244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d9f24-821a-4058-ad8b-01bb160e2410 None admin] RBAC: Authorizing `identity:get_user(user_id=2c53ed74828b46b78bf15befd48ceda3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:24.001924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d9f24-821a-4058-ad8b-01bb160e2410 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:24.011478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 137/262] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:23 2022] GET /identity/v3/users/2c53ed74828b46b78bf15befd48ceda3 => generated 283 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:24.012720 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d9f24-821a-4058-ad8b-01bb160e2410 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:24.076669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:24.150599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9fd38-6632-4c7d-bebc-fb5f57d4be6f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:24.232768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39732bc-0cdc-4f43-a045-dac06da0d86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:24.233889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39732bc-0cdc-4f43-a045-dac06da0d86a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:24.233999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39732bc-0cdc-4f43-a045-dac06da0d86a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:24.238913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39732bc-0cdc-4f43-a045-dac06da0d86a None admin] PATH_INFO: `/v3/users/2c53ed74828b46b78bf15befd48ceda3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.238913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39732bc-0cdc-4f43-a045-dac06da0d86a None admin] RBAC: Authorizing `identity:get_user(user_id=2c53ed74828b46b78bf15befd48ceda3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:24.238913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39732bc-0cdc-4f43-a045-dac06da0d86a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:24.238913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 126/263] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:23 2022] GET /identity/v3/users/2c53ed74828b46b78bf15befd48ceda3 => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:24.243132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:24.251346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39732bc-0cdc-4f43-a045-dac06da0d86a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:24.294567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d9f24-821a-4058-ad8b-01bb160e2410 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:24.373537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b1509-113a-48a0-aa0c-f0d170e9b42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:24.374224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b1509-113a-48a0-aa0c-f0d170e9b42e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:24.374511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b1509-113a-48a0-aa0c-f0d170e9b42e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:24.374793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b1509-113a-48a0-aa0c-f0d170e9b42e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.491133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39732bc-0cdc-4f43-a045-dac06da0d86a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:24.536570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b1509-113a-48a0-aa0c-f0d170e9b42e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:24.538173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b1509-113a-48a0-aa0c-f0d170e9b42e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:24.566860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa18d01-5453-4e95-ac3e-24c5664bed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:24.569040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18d01-5453-4e95-ac3e-24c5664bed8d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:24.569744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18d01-5453-4e95-ac3e-24c5664bed8d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:24.570987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa18d01-5453-4e95-ac3e-24c5664bed8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.575548 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:24.656450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa18d01-5453-4e95-ac3e-24c5664bed8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:24.658774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa18d01-5453-4e95-ac3e-24c5664bed8d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:24.662927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:24.709070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 138/264] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:24.713070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb67fa0-3e2d-4739-be3b-240634b04a48 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:24.713070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb67fa0-3e2d-4739-be3b-240634b04a48 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:24.713642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb67fa0-3e2d-4739-be3b-240634b04a48 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.716156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 139/265] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:24.718539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3df02f-4f4b-48e1-b388-b2535eea33a2 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:24.718834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3df02f-4f4b-48e1-b388-b2535eea33a2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:24.719117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3df02f-4f4b-48e1-b388-b2535eea33a2 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.720428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 140/266] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:24.724950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3df02f-4f4b-48e1-b388-b2535eea33a2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:24.814837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 127/267] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:24.816634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50114d7-dbe4-4068-a1e8-f4152e493672 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:24.817130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50114d7-dbe4-4068-a1e8-f4152e493672 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:24.817607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50114d7-dbe4-4068-a1e8-f4152e493672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.842555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f50114d7-dbe4-4068-a1e8-f4152e493672 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:24.844570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f50114d7-dbe4-4068-a1e8-f4152e493672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:24.847361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3df02f-4f4b-48e1-b388-b2535eea33a2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:24.877879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:24.889836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e14b4-7b1b-4774-9e94-1466e489f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:24.891324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e14b4-7b1b-4774-9e94-1466e489f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:24.894092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e14b4-7b1b-4774-9e94-1466e489f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:24.894092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e14b4-7b1b-4774-9e94-1466e489f6c6 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.894092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e14b4-7b1b-4774-9e94-1466e489f6c6 None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:24.895768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e14b4-7b1b-4774-9e94-1466e489f6c6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:24.898032 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 141/268] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:24 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:24.899875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553ee7-6c1c-489c-944f-ac19d774ad5a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:24.900396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553ee7-6c1c-489c-944f-ac19d774ad5a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:24.900876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e553ee7-6c1c-489c-944f-ac19d774ad5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:24.940003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4e553ee7-6c1c-489c-944f-ac19d774ad5a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:24.940003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4e553ee7-6c1c-489c-944f-ac19d774ad5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:24.940258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 128/269] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:24.940258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50114d7-dbe4-4068-a1e8-f4152e493672 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:24.958127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:25.012792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 142/270] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:25.013760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e553ee7-6c1c-489c-944f-ac19d774ad5a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:25.092204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50114d7-dbe4-4068-a1e8-f4152e493672 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:25.139629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e553ee7-6c1c-489c-944f-ac19d774ad5a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:25.144657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bbe17-ca48-4f78-841f-5d942f237722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:25.145766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bbe17-ca48-4f78-841f-5d942f237722 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.146063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bbe17-ca48-4f78-841f-5d942f237722 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.146411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bbe17-ca48-4f78-841f-5d942f237722 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.147868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bbe17-ca48-4f78-841f-5d942f237722 None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:25.149616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bbe17-ca48-4f78-841f-5d942f237722 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:25.151631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 129/271] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:24 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:25.152408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bbe17-ca48-4f78-841f-5d942f237722 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:25.180047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1e594-9744-4f99-9096-2e380e111a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:25.180717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e594-9744-4f99-9096-2e380e111a78 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.181000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e594-9744-4f99-9096-2e380e111a78 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.181278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e594-9744-4f99-9096-2e380e111a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.182111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1e594-9744-4f99-9096-2e380e111a78 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=366bead620044a59b9ae769237db8568, user.id=2c53ed74828b46b78bf15befd48ceda3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:25.182422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1e594-9744-4f99-9096-2e380e111a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:25.183995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1e594-9744-4f99-9096-2e380e111a78 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:25.191210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 143/272] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:25 2022] GET /identity/v3/role_assignments?user.id=2c53ed74828b46b78bf15befd48ceda3&scope.project.id=366bead620044a59b9ae769237db8568&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:25.194992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:25.203006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a70a89-28a8-4f8d-b87b-ad4cd610f4c5 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.209124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a70a89-28a8-4f8d-b87b-ad4cd610f4c5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.209124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a70a89-28a8-4f8d-b87b-ad4cd610f4c5 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.209124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 144/273] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:25 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:25.209124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a70a89-28a8-4f8d-b87b-ad4cd610f4c5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:25.271012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:25.303922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bbe17-ca48-4f78-841f-5d942f237722 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:25.346406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac7201-1c36-476b-b100-4dc2d5d9b87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:25.347228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac7201-1c36-476b-b100-4dc2d5d9b87d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.347561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac7201-1c36-476b-b100-4dc2d5d9b87d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.347843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac7201-1c36-476b-b100-4dc2d5d9b87d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.363751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac7201-1c36-476b-b100-4dc2d5d9b87d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:25.366784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a70a89-28a8-4f8d-b87b-ad4cd610f4c5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b2ac7201-1c36-476b-b100-4dc2d5d9b87d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:25.372731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:25.373622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:25.374399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 130/274] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:25.375096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac7201-1c36-476b-b100-4dc2d5d9b87d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:25.416490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79217c86-7c1c-4c5c-a921-67877e1f6da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:25.417444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217c86-7c1c-4c5c-a921-67877e1f6da8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.418804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217c86-7c1c-4c5c-a921-67877e1f6da8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.419446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79217c86-7c1c-4c5c-a921-67877e1f6da8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.421000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217c86-7c1c-4c5c-a921-67877e1f6da8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:25.422664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79217c86-7c1c-4c5c-a921-67877e1f6da8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:25.424558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 145/275] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:25.430752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681d3e9-6dba-48ce-a029-554187d93c2d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.431056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681d3e9-6dba-48ce-a029-554187d93c2d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.431449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681d3e9-6dba-48ce-a029-554187d93c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.453619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4681d3e9-6dba-48ce-a029-554187d93c2d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:25.454767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4681d3e9-6dba-48ce-a029-554187d93c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:25.502887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:25.514871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac7201-1c36-476b-b100-4dc2d5d9b87d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:25.542057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 146/276] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:25.543494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3cc5a-5512-443b-8887-74ed05969dbf None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.543787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3cc5a-5512-443b-8887-74ed05969dbf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.544066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3cc5a-5512-443b-8887-74ed05969dbf None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.545340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 147/277] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:25.546031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3cc5a-5512-443b-8887-74ed05969dbf None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:25.553915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:25.554574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.554863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.555144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.556048 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] RBAC: Adding query filter params (role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=366bead620044a59b9ae769237db8568, user.id=2c53ed74828b46b78bf15befd48ceda3) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:25.556384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:25.557862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:25.564219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 131/278] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:25 2022] GET /identity/v3/role_assignments?user.id=2c53ed74828b46b78bf15befd48ceda3&scope.project.id=366bead620044a59b9ae769237db8568&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:25.565000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:25.591065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:25.693758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e3cc5a-5512-443b-8887-74ed05969dbf None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:25.707477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bb6b3-ebfa-411b-92ca-efdee4c95d87 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:25.749623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d78f60-b342-45bb-a7af-e149623973af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:25.750589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d78f60-b342-45bb-a7af-e149623973af None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.750589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d78f60-b342-45bb-a7af-e149623973af None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.750855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d78f60-b342-45bb-a7af-e149623973af None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.752029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d78f60-b342-45bb-a7af-e149623973af None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:25.752029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d78f60-b342-45bb-a7af-e149623973af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:25.753314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d78f60-b342-45bb-a7af-e149623973af None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:25.763104 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f68a9-562a-4361-b68c-5063d25eadc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:25.763819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f68a9-562a-4361-b68c-5063d25eadc0 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.764110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f68a9-562a-4361-b68c-5063d25eadc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.764391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f68a9-562a-4361-b68c-5063d25eadc0 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.765503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 148/279] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:25.765611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f68a9-562a-4361-b68c-5063d25eadc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:25.766889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f68a9-562a-4361-b68c-5063d25eadc0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:25.774959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38641336-1cf9-46fc-b823-165b6d85dc7a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:25.774959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38641336-1cf9-46fc-b823-165b6d85dc7a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:25.774959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38641336-1cf9-46fc-b823-165b6d85dc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:25.795159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-38641336-1cf9-46fc-b823-165b6d85dc7a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:25.796488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-38641336-1cf9-46fc-b823-165b6d85dc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:25.867339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 149/280] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:25.867339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38641336-1cf9-46fc-b823-165b6d85dc7a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:25.891856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 132/281] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:25 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:25.891856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f68a9-562a-4361-b68c-5063d25eadc0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:25.903169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:25.907680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:26.000095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38641336-1cf9-46fc-b823-165b6d85dc7a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:26.037989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f68a9-562a-4361-b68c-5063d25eadc0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:26.049320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806882ad-9568-49de-be32-731855287990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:26.050356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806882ad-9568-49de-be32-731855287990 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:26.050821 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806882ad-9568-49de-be32-731855287990 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:26.051250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806882ad-9568-49de-be32-731855287990 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:26.066993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806882ad-9568-49de-be32-731855287990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-806882ad-9568-49de-be32-731855287990 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:26.075239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:26.076168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:26.076927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 150/282] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:26.080668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806882ad-9568-49de-be32-731855287990 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:26.098887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b12892-ff5c-45bd-affb-6ea43ee8ba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:26.100010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b12892-ff5c-45bd-affb-6ea43ee8ba6a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:26.100368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b12892-ff5c-45bd-affb-6ea43ee8ba6a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:26.100656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b12892-ff5c-45bd-affb-6ea43ee8ba6a None admin] PATH_INFO: `/v3/users/716f667a20ef47628c366b225cf71a55` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:26.131935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b12892-ff5c-45bd-affb-6ea43ee8ba6a None admin] RBAC: Authorizing `identity:get_user(user_id=716f667a20ef47628c366b225cf71a55)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:26.136695 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b12892-ff5c-45bd-affb-6ea43ee8ba6a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:26.144689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 133/283] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:25 2022] GET /identity/v3/users/716f667a20ef47628c366b225cf71a55 => generated 293 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:26.144689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b12892-ff5c-45bd-affb-6ea43ee8ba6a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:26.220669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:26.244710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:26.251894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806882ad-9568-49de-be32-731855287990 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:26.307085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b12892-ff5c-45bd-affb-6ea43ee8ba6a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:26.322734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a10dcf-2905-4f67-a357-05038ab3dc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:26.323668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a10dcf-2905-4f67-a357-05038ab3dc18 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:26.323891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a10dcf-2905-4f67-a357-05038ab3dc18 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:26.324147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a10dcf-2905-4f67-a357-05038ab3dc18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:26.343810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a10dcf-2905-4f67-a357-05038ab3dc18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:26.349251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:26.350802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:26.351456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:26.351942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:26.352972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:26.353480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:26.355079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b9a10dcf-2905-4f67-a357-05038ab3dc18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:26.357102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:26.357955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:26.358746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 151/284] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:26.359401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a10dcf-2905-4f67-a357-05038ab3dc18 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:26.368102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 134/285] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:26.369254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:26.503543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6542302-fe32-43e2-80ee-ca0fc13368f3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:26.550216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:26.554102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a10dcf-2905-4f67-a357-05038ab3dc18 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:26.571244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:26.571244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:26.571244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:26.571244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:26.571807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:26.571807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:26.573121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:26.582327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 135/286] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:26.583141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:26.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:26.600912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:26.601201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:26.601483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:26.604965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] RBAC: Adding query filter params (role.id=0892f8baef1c4b61b13187ba945d2ad5, user.id=716f667a20ef47628c366b225cf71a55, scope.system=all) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:26.605152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:26.606642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:26.616242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 152/287] 10.222.0.98 () {56 vars in 1308 bytes} [Fri Jul 15 15:32:26 2022] GET /identity/v3/role_assignments?user.id=716f667a20ef47628c366b225cf71a55&scope.system=all&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:26.618138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:26.648601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:26.809630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc371dce-5d41-4d32-bf8f-c9584fceedf8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:26.849145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a193407-21d8-4e8d-994d-88144dcfc7a4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:26.864376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821604-81e2-42c7-abaf-b99a090ee621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:26.866100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821604-81e2-42c7-abaf-b99a090ee621 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:26.866418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821604-81e2-42c7-abaf-b99a090ee621 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:26.866772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821604-81e2-42c7-abaf-b99a090ee621 None admin] PATH_INFO: `/v3/users/a2901d3ff8184dff9e6c0a54146f6809` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:26.868045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1821604-81e2-42c7-abaf-b99a090ee621 None admin] RBAC: Authorizing `identity:get_user(user_id=a2901d3ff8184dff9e6c0a54146f6809)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:26.869520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1821604-81e2-42c7-abaf-b99a090ee621 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:26.871886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 136/288] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:26 2022] GET /identity/v3/users/a2901d3ff8184dff9e6c0a54146f6809 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:26.877554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821604-81e2-42c7-abaf-b99a090ee621 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:26.900771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c43d18-29b2-4563-b397-2560e46cc1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:26.901637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c43d18-29b2-4563-b397-2560e46cc1f1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:26.901920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c43d18-29b2-4563-b397-2560e46cc1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:26.902198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c43d18-29b2-4563-b397-2560e46cc1f1 None admin] PATH_INFO: `/v3/users/3c9b6834a7774691aa6b08b7a53ea7db` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:26.904675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c43d18-29b2-4563-b397-2560e46cc1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9b6834a7774691aa6b08b7a53ea7db)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:26.906128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c43d18-29b2-4563-b397-2560e46cc1f1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:26.907840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 153/289] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:26 2022] GET /identity/v3/users/3c9b6834a7774691aa6b08b7a53ea7db => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:26.916142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c43d18-29b2-4563-b397-2560e46cc1f1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:27.004104 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:27.059629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:27.094864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c43d18-29b2-4563-b397-2560e46cc1f1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:27.101776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821604-81e2-42c7-abaf-b99a090ee621 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:27.196071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b647a-177a-42da-b006-60ab7c547cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:27.197087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bef4d-8877-421e-9cd6-b48d298ae76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:27.197962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bef4d-8877-421e-9cd6-b48d298ae76c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:27.198256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bef4d-8877-421e-9cd6-b48d298ae76c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:27.198535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bef4d-8877-421e-9cd6-b48d298ae76c None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:27.199431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b647a-177a-42da-b006-60ab7c547cd5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:27.199720 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b647a-177a-42da-b006-60ab7c547cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:27.199999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b647a-177a-42da-b006-60ab7c547cd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:27.201245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bef4d-8877-421e-9cd6-b48d298ae76c None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:27.202704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bef4d-8877-421e-9cd6-b48d298ae76c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:27.204460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 137/290] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:26 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:27.215947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bef4d-8877-421e-9cd6-b48d298ae76c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:27.342579 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b647a-177a-42da-b006-60ab7c547cd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:27.345839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b647a-177a-42da-b006-60ab7c547cd5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:27.475961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:27.508904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bef4d-8877-421e-9cd6-b48d298ae76c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:27.540604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 154/291] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:27.543847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b647a-177a-42da-b006-60ab7c547cd5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:27.575658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:27.591110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f8a90-7552-440a-ae10-94033b1e29ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:27.592046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f8a90-7552-440a-ae10-94033b1e29ab None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:27.592486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f8a90-7552-440a-ae10-94033b1e29ab None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:27.592584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f8a90-7552-440a-ae10-94033b1e29ab None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568/users/a2901d3ff8184dff9e6c0a54146f6809/roles/3555f68f0fca46018de9022f06dcdc90` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:27.595738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f8a90-7552-440a-ae10-94033b1e29ab None admin] RBAC: Authorizing `identity:create_grant(project_id=366bead620044a59b9ae769237db8568, user_id=a2901d3ff8184dff9e6c0a54146f6809, role_id=3555f68f0fca46018de9022f06dcdc90)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:27.597286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f8a90-7552-440a-ae10-94033b1e29ab None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:27.683557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b647a-177a-42da-b006-60ab7c547cd5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:27.697701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 138/292] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:32:27 2022] PUT /identity/v3/projects/366bead620044a59b9ae769237db8568/users/a2901d3ff8184dff9e6c0a54146f6809/roles/3555f68f0fca46018de9022f06dcdc90 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:27.770335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f70d0-76bd-4a10-9783-856ebda8d02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:27.771268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f70d0-76bd-4a10-9783-856ebda8d02e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:27.771675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f70d0-76bd-4a10-9783-856ebda8d02e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:27.771958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f70d0-76bd-4a10-9783-856ebda8d02e None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:27.773194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f70d0-76bd-4a10-9783-856ebda8d02e None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:27.774942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f70d0-76bd-4a10-9783-856ebda8d02e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:27.776691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 155/293] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:27 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:27.783328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f8a90-7552-440a-ae10-94033b1e29ab None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:28.047439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f8a90-7552-440a-ae10-94033b1e29ab None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:28.086447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db531ee7-b981-4c63-bebe-ee7018b42a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:28.087117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db531ee7-b981-4c63-bebe-ee7018b42a9a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:28.087502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db531ee7-b981-4c63-bebe-ee7018b42a9a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:28.087786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db531ee7-b981-4c63-bebe-ee7018b42a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:28.088636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db531ee7-b981-4c63-bebe-ee7018b42a9a None admin] RBAC: Adding query filter params (role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=366bead620044a59b9ae769237db8568, user.id=3c9b6834a7774691aa6b08b7a53ea7db) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:28.088953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db531ee7-b981-4c63-bebe-ee7018b42a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:28.090653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db531ee7-b981-4c63-bebe-ee7018b42a9a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:28.099393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 139/294] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:27 2022] GET /identity/v3/role_assignments?user.id=3c9b6834a7774691aa6b08b7a53ea7db&scope.project.id=366bead620044a59b9ae769237db8568&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:28.438878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035228-1fe8-425e-aa69-b07d5cf853a0 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:28.438878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035228-1fe8-425e-aa69-b07d5cf853a0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:28.438878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f035228-1fe8-425e-aa69-b07d5cf853a0 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:28.438878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 156/295] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:28.445556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b3dbc-571c-4705-b062-5c7c7f65f755 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:28.445941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b3dbc-571c-4705-b062-5c7c7f65f755 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:28.446296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b3dbc-571c-4705-b062-5c7c7f65f755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:28.465009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3c6b3dbc-571c-4705-b062-5c7c7f65f755 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:28.466489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3c6b3dbc-571c-4705-b062-5c7c7f65f755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:28.523127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:28.545365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 140/296] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:28.570722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f035228-1fe8-425e-aa69-b07d5cf853a0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:28.588739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791838e8-7f73-4cc2-abea-8e5766be08f1 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:28.589347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791838e8-7f73-4cc2-abea-8e5766be08f1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:28.589831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791838e8-7f73-4cc2-abea-8e5766be08f1 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:28.591946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 141/297] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:28.597318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3785c7-3511-4444-a686-47f98f7482fa None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:28.597452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3785c7-3511-4444-a686-47f98f7482fa None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:28.597718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3785c7-3511-4444-a686-47f98f7482fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:28.621762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4f3785c7-3511-4444-a686-47f98f7482fa None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:28.624446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4f3785c7-3511-4444-a686-47f98f7482fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:28.648179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:28.714942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 142/298] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:28.751284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3785c7-3511-4444-a686-47f98f7482fa None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:28.754599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f035228-1fe8-425e-aa69-b07d5cf853a0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:28.796013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094c664-2a5f-42ef-8b0f-78512dc6d511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:28.796884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094c664-2a5f-42ef-8b0f-78512dc6d511 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:28.797095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094c664-2a5f-42ef-8b0f-78512dc6d511 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:28.797374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094c664-2a5f-42ef-8b0f-78512dc6d511 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:28.809711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094c664-2a5f-42ef-8b0f-78512dc6d511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0094c664-2a5f-42ef-8b0f-78512dc6d511 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:28.818343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:28.819373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:28.820710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 157/299] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:28.826028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094c664-2a5f-42ef-8b0f-78512dc6d511 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:28.890558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3785c7-3511-4444-a686-47f98f7482fa None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:28.908461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:28.937988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18482223-c3ee-44e9-8e97-e068f8e31fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:28.941532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18482223-c3ee-44e9-8e97-e068f8e31fc4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:28.942024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18482223-c3ee-44e9-8e97-e068f8e31fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:28.942493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18482223-c3ee-44e9-8e97-e068f8e31fc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:28.945458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18482223-c3ee-44e9-8e97-e068f8e31fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:28.948275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18482223-c3ee-44e9-8e97-e068f8e31fc4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:28.950315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 143/300] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:28.955944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18482223-c3ee-44e9-8e97-e068f8e31fc4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:28.992642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094c664-2a5f-42ef-8b0f-78512dc6d511 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:28.995136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:29.042158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:29.042158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:29.042158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:29.042158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:29.042546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:29.042546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:29.042546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:29.051420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 158/301] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:29.058672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:29.151329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18482223-c3ee-44e9-8e97-e068f8e31fc4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:29.211671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b165da-cfcd-4345-9b15-c66d51f0fab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:29.212383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b165da-cfcd-4345-9b15-c66d51f0fab8 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:29.212670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b165da-cfcd-4345-9b15-c66d51f0fab8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:29.212947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b165da-cfcd-4345-9b15-c66d51f0fab8 None admin] PATH_INFO: `/v3/groups` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:29.215320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b165da-cfcd-4345-9b15-c66d51f0fab8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:29.215320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b165da-cfcd-4345-9b15-c66d51f0fab8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:29.217796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a902e-baf5-4634-8eb0-d81c6036a0d4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:29.261058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b157300-043d-4e8f-9d82-7b9066936822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:29.261058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157300-043d-4e8f-9d82-7b9066936822 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:29.261058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157300-043d-4e8f-9d82-7b9066936822 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:29.261058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157300-043d-4e8f-9d82-7b9066936822 None admin] PATH_INFO: `/v3/users/9c439831936842d3a5ba93162dd84eb9` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:29.272925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b157300-043d-4e8f-9d82-7b9066936822 None admin] RBAC: Authorizing `identity:get_user(user_id=9c439831936842d3a5ba93162dd84eb9)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:29.274702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b157300-043d-4e8f-9d82-7b9066936822 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:29.276669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 159/302] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:29 2022] GET /identity/v3/users/9c439831936842d3a5ba93162dd84eb9 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:29.277799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e948bc-1f7b-4f69-8486-6cde6c94bd61 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:29.278098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e948bc-1f7b-4f69-8486-6cde6c94bd61 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:29.278845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e948bc-1f7b-4f69-8486-6cde6c94bd61 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:29.283390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 160/303] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:29.283390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e948bc-1f7b-4f69-8486-6cde6c94bd61 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:29.300309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 144/304] 10.222.0.98 () {60 vars in 1134 bytes} [Fri Jul 15 15:32:28 2022] POST /identity/v3/groups => generated 229 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:32:29.300309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a679d2-4fbc-4117-8c48-1e8d272e0808 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:29.300309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a679d2-4fbc-4117-8c48-1e8d272e0808 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:29.300309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a679d2-4fbc-4117-8c48-1e8d272e0808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:29.316401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:29.340458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-26a679d2-4fbc-4117-8c48-1e8d272e0808 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:29.342160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-26a679d2-4fbc-4117-8c48-1e8d272e0808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:29.434039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e948bc-1f7b-4f69-8486-6cde6c94bd61 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:29.436534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:29.502125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:29.503241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:29.503241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:29.503883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:29.505229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] RBAC: Adding query filter params (role.id=3555f68f0fca46018de9022f06dcdc90, user.id=9c439831936842d3a5ba93162dd84eb9, scope.system=all) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:29.505229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:29.506938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:29.510630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 145/305] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:29.523426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 161/306] 10.222.0.98 () {56 vars in 1308 bytes} [Fri Jul 15 15:32:29 2022] GET /identity/v3/role_assignments?user.id=9c439831936842d3a5ba93162dd84eb9&scope.system=all&role.id=3555f68f0fca46018de9022f06dcdc90 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:29.541793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a679d2-4fbc-4117-8c48-1e8d272e0808 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:29.666371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a679d2-4fbc-4117-8c48-1e8d272e0808 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:29.706272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb4ef8-08e0-46bb-9f9d-8f2ec2f58f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:29.707430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4ef8-08e0-46bb-9f9d-8f2ec2f58f4a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:29.707890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4ef8-08e0-46bb-9f9d-8f2ec2f58f4a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:29.708349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4ef8-08e0-46bb-9f9d-8f2ec2f58f4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:29.720571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb4ef8-08e0-46bb-9f9d-8f2ec2f58f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-94eb4ef8-08e0-46bb-9f9d-8f2ec2f58f4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:29.729263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:29.730389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:29.731399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:29.733669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 146/307] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:29.740584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:29.884022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:29.906847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0e709-b087-4e50-af8a-e3094d698fc7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:29.968134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f29a2-92dd-4eb9-a35b-ef2507ca3f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:29.968845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f29a2-92dd-4eb9-a35b-ef2507ca3f17 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:29.969137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f29a2-92dd-4eb9-a35b-ef2507ca3f17 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:29.969417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f29a2-92dd-4eb9-a35b-ef2507ca3f17 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:29.970175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f29a2-92dd-4eb9-a35b-ef2507ca3f17 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:29.970481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f29a2-92dd-4eb9-a35b-ef2507ca3f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:29.971965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f29a2-92dd-4eb9-a35b-ef2507ca3f17 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:29.983036 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 162/308] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:29.989687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb4ef8-08e0-46bb-9f9d-8f2ec2f58f4a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:30.034772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:30.127254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb4ef8-08e0-46bb-9f9d-8f2ec2f58f4a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:30.182495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ba61d-d459-4d30-a70e-b5d1e445bfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:30.186011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ba61d-d459-4d30-a70e-b5d1e445bfa8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:30.186987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ba61d-d459-4d30-a70e-b5d1e445bfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:30.187503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ba61d-d459-4d30-a70e-b5d1e445bfa8 None admin] PATH_INFO: `/v3/users/716f667a20ef47628c366b225cf71a55` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:30.189002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ba61d-d459-4d30-a70e-b5d1e445bfa8 None admin] RBAC: Authorizing `identity:get_user(user_id=716f667a20ef47628c366b225cf71a55)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:30.190716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ba61d-d459-4d30-a70e-b5d1e445bfa8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:30.193958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 147/309] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:29 2022] GET /identity/v3/users/716f667a20ef47628c366b225cf71a55 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:30.201638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ba61d-d459-4d30-a70e-b5d1e445bfa8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:30.202197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c898a87-6ba8-460d-bd59-10da934a1d23 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:30.202520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c898a87-6ba8-460d-bd59-10da934a1d23 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:30.202760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c898a87-6ba8-460d-bd59-10da934a1d23 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:30.204322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 163/310] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:30.210265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa46af-2b78-4be6-a5e6-b69bd24a8e70 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:30.210533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa46af-2b78-4be6-a5e6-b69bd24a8e70 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:30.210816 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa46af-2b78-4be6-a5e6-b69bd24a8e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:30.288329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b9aa46af-2b78-4be6-a5e6-b69bd24a8e70 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:30.289572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b9aa46af-2b78-4be6-a5e6-b69bd24a8e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:30.315562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:30.322894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ba61d-d459-4d30-a70e-b5d1e445bfa8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:30.360595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed55be3-ced1-412c-8a4f-e77b671e1939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:30.361871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55be3-ced1-412c-8a4f-e77b671e1939 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:30.362326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55be3-ced1-412c-8a4f-e77b671e1939 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:30.362760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed55be3-ced1-412c-8a4f-e77b671e1939 None admin] PATH_INFO: `/v3/system/users/716f667a20ef47628c366b225cf71a55/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:30.364555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed55be3-ced1-412c-8a4f-e77b671e1939 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=716f667a20ef47628c366b225cf71a55, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:30.366642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed55be3-ced1-412c-8a4f-e77b671e1939 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:30.450327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 164/311] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:30.478357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa46af-2b78-4be6-a5e6-b69bd24a8e70 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:30.505323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 148/312] 10.222.0.98 () {58 vars in 1334 bytes} [Fri Jul 15 15:32:30 2022] PUT /identity/v3/system/users/716f667a20ef47628c366b225cf71a55/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:30.583247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:30.631876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa46af-2b78-4be6-a5e6-b69bd24a8e70 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:30.708117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb67fe-4bd7-4a66-897b-b1b2404e4e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:30.709009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb67fe-4bd7-4a66-897b-b1b2404e4e5b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:30.709301 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb67fe-4bd7-4a66-897b-b1b2404e4e5b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:30.709586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb67fe-4bd7-4a66-897b-b1b2404e4e5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:30.720445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb67fe-4bd7-4a66-897b-b1b2404e4e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d0cb67fe-4bd7-4a66-897b-b1b2404e4e5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:30.729694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:30.730689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:30.731731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 165/313] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:30.745504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed55be3-ced1-412c-8a4f-e77b671e1939 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:30.869348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed55be3-ced1-412c-8a4f-e77b671e1939 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:30.907179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:30.908191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:30.908567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:30.908951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:30.909823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:30.910208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:30.911701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:30.920999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 149/314] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:30.928557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb67fe-4bd7-4a66-897b-b1b2404e4e5b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:31.147196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb67fe-4bd7-4a66-897b-b1b2404e4e5b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:31.208461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afab0d-0515-434e-85a6-7526f287bea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:31.209380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afab0d-0515-434e-85a6-7526f287bea8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:31.209662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afab0d-0515-434e-85a6-7526f287bea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:31.209935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afab0d-0515-434e-85a6-7526f287bea8 None admin] PATH_INFO: `/v3/users/a2901d3ff8184dff9e6c0a54146f6809` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:31.211171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afab0d-0515-434e-85a6-7526f287bea8 None admin] RBAC: Authorizing `identity:get_user(user_id=a2901d3ff8184dff9e6c0a54146f6809)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:31.213173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afab0d-0515-434e-85a6-7526f287bea8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:31.216058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 166/315] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:30 2022] GET /identity/v3/users/a2901d3ff8184dff9e6c0a54146f6809 => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:31.241013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:31.304168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:31.471686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852657a-d9ea-4b1e-bac8-d38d99e4cabd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:31.574195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813774aa-5455-4502-8a34-7dfe289deab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:31.574965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813774aa-5455-4502-8a34-7dfe289deab9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:31.575072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813774aa-5455-4502-8a34-7dfe289deab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:31.575930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813774aa-5455-4502-8a34-7dfe289deab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:31.742718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595961-7c21-4662-9a97-28d9d62200e7 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:31.743267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595961-7c21-4662-9a97-28d9d62200e7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:31.743267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595961-7c21-4662-9a97-28d9d62200e7 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:31.745747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 167/316] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:31.753640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d9b5d-4ddd-44c8-b88e-ab98c3a953b5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:31.754475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d9b5d-4ddd-44c8-b88e-ab98c3a953b5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:31.755188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d9b5d-4ddd-44c8-b88e-ab98c3a953b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:31.776711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:31.783239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-da9d9b5d-4ddd-44c8-b88e-ab98c3a953b5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:31.785139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-da9d9b5d-4ddd-44c8-b88e-ab98c3a953b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:31.810913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813774aa-5455-4502-8a34-7dfe289deab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:31.812435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813774aa-5455-4502-8a34-7dfe289deab9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:31.860106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:31.872384 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 168/317] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:31.874668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03970b23-4ce8-4007-a388-b8de56a57076 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:31.875054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03970b23-4ce8-4007-a388-b8de56a57076 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:31.875446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03970b23-4ce8-4007-a388-b8de56a57076 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:31.877682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 169/318] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:31.890968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aedda-85ed-46fc-984d-c5e610478f00 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:31.892843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aedda-85ed-46fc-984d-c5e610478f00 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:31.893722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aedda-85ed-46fc-984d-c5e610478f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:31.921277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7f7aedda-85ed-46fc-984d-c5e610478f00 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:31.923457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7f7aedda-85ed-46fc-984d-c5e610478f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:31.965404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 150/319] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:31.966694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813774aa-5455-4502-8a34-7dfe289deab9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:32.043381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 170/320] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:32.048408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7aedda-85ed-46fc-984d-c5e610478f00 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:32.076332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813774aa-5455-4502-8a34-7dfe289deab9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:32.101857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:32.113171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1f98-4c4c-442a-a343-cc96074024f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:32.113960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1f98-4c4c-442a-a343-cc96074024f8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.114246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1f98-4c4c-442a-a343-cc96074024f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.114522 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1f98-4c4c-442a-a343-cc96074024f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.115409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:32.124942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a1f98-4c4c-442a-a343-cc96074024f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ce3a1f98-4c4c-442a-a343-cc96074024f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:32.132693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:32.133580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:32.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 151/321] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:32.135009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1f98-4c4c-442a-a343-cc96074024f8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:32.156373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7aedda-85ed-46fc-984d-c5e610478f00 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:32.192398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd5460-a474-4139-a575-142582bb4784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:32.193763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd5460-a474-4139-a575-142582bb4784 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.194055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd5460-a474-4139-a575-142582bb4784 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.194340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd5460-a474-4139-a575-142582bb4784 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.195547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd5460-a474-4139-a575-142582bb4784 None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:32.197004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd5460-a474-4139-a575-142582bb4784 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:32.198965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 171/322] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:32 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:32.199638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd5460-a474-4139-a575-142582bb4784 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:32.257221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a1f98-4c4c-442a-a343-cc96074024f8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:32.293183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb989e-4f10-4d21-b63f-85adfa5be1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:32.294033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb989e-4f10-4d21-b63f-85adfa5be1c2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.294265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb989e-4f10-4d21-b63f-85adfa5be1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.294541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb989e-4f10-4d21-b63f-85adfa5be1c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.295743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb989e-4f10-4d21-b63f-85adfa5be1c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:32.297198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb989e-4f10-4d21-b63f-85adfa5be1c2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:32.298987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 152/323] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:32.304524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb989e-4f10-4d21-b63f-85adfa5be1c2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:32.307643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd5460-a474-4139-a575-142582bb4784 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:32.344358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:32.344985 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.345273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.345555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.346308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:32.346621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:32.348052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:32.350002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:32.357549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 172/324] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:32.362959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:32.364455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:32.430036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb989e-4f10-4d21-b63f-85adfa5be1c2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:32.465582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:32.466240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.466527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.466811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.467669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] RBAC: Adding query filter params (role.id=3555f68f0fca46018de9022f06dcdc90, scope.project.id=366bead620044a59b9ae769237db8568, user.id=a2901d3ff8184dff9e6c0a54146f6809) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:32.468010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:32.469437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:32.472412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7fe64-e503-49b2-ac35-059cad99b8d5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:32.478447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 153/325] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:32 2022] GET /identity/v3/role_assignments?user.id=a2901d3ff8184dff9e6c0a54146f6809&scope.project.id=366bead620044a59b9ae769237db8568&role.id=3555f68f0fca46018de9022f06dcdc90 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:32.478447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:32.515149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32cc2c-2e5f-4594-862b-eb72ed15dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:32.515904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32cc2c-2e5f-4594-862b-eb72ed15dcdd None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.516171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32cc2c-2e5f-4594-862b-eb72ed15dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.516456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32cc2c-2e5f-4594-862b-eb72ed15dcdd None admin] PATH_INFO: `/v3/groups` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.517157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32cc2c-2e5f-4594-862b-eb72ed15dcdd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:32.518629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32cc2c-2e5f-4594-862b-eb72ed15dcdd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:32.606205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b654c1-e19a-4a43-83a7-93732fcab47d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:32.623338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 173/326] 10.222.0.98 () {60 vars in 1134 bytes} [Fri Jul 15 15:32:32 2022] POST /identity/v3/groups => generated 226 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:32.632288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:32.660941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a9d5-3b0b-4498-9930-e4ca9d63b254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:32.661846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a9d5-3b0b-4498-9930-e4ca9d63b254 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.662187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a9d5-3b0b-4498-9930-e4ca9d63b254 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.662509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a9d5-3b0b-4498-9930-e4ca9d63b254 None admin] PATH_INFO: `/v3/users/9c439831936842d3a5ba93162dd84eb9` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.664361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a9d5-3b0b-4498-9930-e4ca9d63b254 None admin] RBAC: Authorizing `identity:get_user(user_id=9c439831936842d3a5ba93162dd84eb9)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:32.665550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a9d5-3b0b-4498-9930-e4ca9d63b254 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:32.668510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 154/327] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:32 2022] GET /identity/v3/users/9c439831936842d3a5ba93162dd84eb9 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:32.673431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32cc2c-2e5f-4594-862b-eb72ed15dcdd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:32.745688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:32.795364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32cc2c-2e5f-4594-862b-eb72ed15dcdd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:32.808965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc45a-f5ca-421f-b81f-70d83d8ef236 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.809259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc45a-f5ca-421f-b81f-70d83d8ef236 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.809713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc45a-f5ca-421f-b81f-70d83d8ef236 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.811238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 155/328] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:32.816389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4b7b-066a-46e0-b213-cbc0e1067afb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.817755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4b7b-066a-46e0-b213-cbc0e1067afb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.817755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e4b7b-066a-46e0-b213-cbc0e1067afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.831411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c4d2d-8585-4cbd-a531-2bcc860bee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:32.832291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c4d2d-8585-4cbd-a531-2bcc860bee54 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:32.832582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c4d2d-8585-4cbd-a531-2bcc860bee54 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:32.832864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c4d2d-8585-4cbd-a531-2bcc860bee54 None admin] PATH_INFO: `/v3/system/users/9c439831936842d3a5ba93162dd84eb9/roles/3555f68f0fca46018de9022f06dcdc90` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:32.834927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c4d2d-8585-4cbd-a531-2bcc860bee54 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9c439831936842d3a5ba93162dd84eb9, role_id=3555f68f0fca46018de9022f06dcdc90)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:32.836415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c4d2d-8585-4cbd-a531-2bcc860bee54 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:32.840642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-473e4b7b-066a-46e0-b213-cbc0e1067afb None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:32.842206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-473e4b7b-066a-46e0-b213-cbc0e1067afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:32.909391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 174/329] 10.222.0.98 () {58 vars in 1334 bytes} [Fri Jul 15 15:32:32 2022] PUT /identity/v3/system/users/9c439831936842d3a5ba93162dd84eb9/roles/3555f68f0fca46018de9022f06dcdc90 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:32.915489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 156/330] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:32.939509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c4d2d-8585-4cbd-a531-2bcc860bee54 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:33.076259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c4d2d-8585-4cbd-a531-2bcc860bee54 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:33.099282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:33.116062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9056a8-5680-43a1-8676-3f90dff24cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:33.116874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9056a8-5680-43a1-8676-3f90dff24cbe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:33.117161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9056a8-5680-43a1-8676-3f90dff24cbe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:33.117445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9056a8-5680-43a1-8676-3f90dff24cbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:33.132184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9056a8-5680-43a1-8676-3f90dff24cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-3d9056a8-5680-43a1-8676-3f90dff24cbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:33.142427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:33.143339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:33.144112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 175/331] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:33.148306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e4b7b-066a-46e0-b213-cbc0e1067afb None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:33.274755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e4b7b-066a-46e0-b213-cbc0e1067afb None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:33.291332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:33.326763 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:33.327903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:33.328383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:33.330050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:33.331141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:33.332673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:33.334828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:33.350513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 157/332] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:33.355719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9056a8-5680-43a1-8676-3f90dff24cbe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:33.471091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9056a8-5680-43a1-8676-3f90dff24cbe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:33.507123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93549f36-e006-46c5-8600-220c18d3c0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:33.508039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549f36-e006-46c5-8600-220c18d3c0c8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:33.508321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549f36-e006-46c5-8600-220c18d3c0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:33.508596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549f36-e006-46c5-8600-220c18d3c0c8 None admin] PATH_INFO: `/v3/users/716f667a20ef47628c366b225cf71a55` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:33.509790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93549f36-e006-46c5-8600-220c18d3c0c8 None admin] RBAC: Authorizing `identity:get_user(user_id=716f667a20ef47628c366b225cf71a55)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:33.511257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93549f36-e006-46c5-8600-220c18d3c0c8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:33.513235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 176/333] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:33 2022] GET /identity/v3/users/716f667a20ef47628c366b225cf71a55 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:33.518842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:33.644300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376658a-87f3-46b4-9419-7fef1572aabc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:33.690339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4264d-3a62-401f-82ba-7bf60a085807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:33.691071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4264d-3a62-401f-82ba-7bf60a085807 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:33.691193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4264d-3a62-401f-82ba-7bf60a085807 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:33.691510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4264d-3a62-401f-82ba-7bf60a085807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:33.693394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4264d-3a62-401f-82ba-7bf60a085807 None admin] RBAC: Adding query filter params (role.id=0892f8baef1c4b61b13187ba945d2ad5, user.id=716f667a20ef47628c366b225cf71a55, scope.system=all) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:33.693394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4264d-3a62-401f-82ba-7bf60a085807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:33.694770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4264d-3a62-401f-82ba-7bf60a085807 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:33.707885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 158/334] 10.222.0.98 () {56 vars in 1308 bytes} [Fri Jul 15 15:32:33 2022] GET /identity/v3/role_assignments?user.id=716f667a20ef47628c366b225cf71a55&scope.system=all&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:35.196091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d4d70-458d-4d83-b425-8139af12cdf2 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.196639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d4d70-458d-4d83-b425-8139af12cdf2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.196917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d4d70-458d-4d83-b425-8139af12cdf2 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.198647 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 177/335] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:35.204164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bcb46-6331-4caf-ac0c-5d65f180c02b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.204574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bcb46-6331-4caf-ac0c-5d65f180c02b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.204794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bcb46-6331-4caf-ac0c-5d65f180c02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.222570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4d2bcb46-6331-4caf-ac0c-5d65f180c02b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:35.224770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4d2bcb46-6331-4caf-ac0c-5d65f180c02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:35.252828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:35.313832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 159/336] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:35.352566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d4d70-458d-4d83-b425-8139af12cdf2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:35.417631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:35.466066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7538193-f480-4478-9f0c-025312b6ae88 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.466383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7538193-f480-4478-9f0c-025312b6ae88 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.466637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7538193-f480-4478-9f0c-025312b6ae88 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.468203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 160/337] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:35.475274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf5744-f404-40ba-a12e-999e51cf4e33 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.475274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf5744-f404-40ba-a12e-999e51cf4e33 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.475274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf5744-f404-40ba-a12e-999e51cf4e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.497062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fcbf5744-f404-40ba-a12e-999e51cf4e33 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:35.498664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fcbf5744-f404-40ba-a12e-999e51cf4e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:35.520911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d4d70-458d-4d83-b425-8139af12cdf2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:35.580462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2539c9f-ac35-4425-a919-0090e89a974c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:35.581275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2539c9f-ac35-4425-a919-0090e89a974c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.581563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2539c9f-ac35-4425-a919-0090e89a974c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.581846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2539c9f-ac35-4425-a919-0090e89a974c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.596602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2539c9f-ac35-4425-a919-0090e89a974c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d2539c9f-ac35-4425-a919-0090e89a974c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:35.605201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:35.606051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:35.606803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 178/338] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:35.612523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2539c9f-ac35-4425-a919-0090e89a974c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:35.651185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 161/339] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:35.691376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219554d-00fb-422f-8994-9f101cdb97ac None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.691695 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219554d-00fb-422f-8994-9f101cdb97ac None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.692070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219554d-00fb-422f-8994-9f101cdb97ac None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.694839 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 162/340] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:35.698548 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219554d-00fb-422f-8994-9f101cdb97ac None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:35.727272 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2539c9f-ac35-4425-a919-0090e89a974c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:35.738285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:35.773241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c1a8a-1a63-4708-bb9d-b2681592fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:35.775850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1a8a-1a63-4708-bb9d-b2681592fe75 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.777466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1a8a-1a63-4708-bb9d-b2681592fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.777945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c1a8a-1a63-4708-bb9d-b2681592fe75 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.779946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1a8a-1a63-4708-bb9d-b2681592fe75 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:35.780449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1a8a-1a63-4708-bb9d-b2681592fe75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:35.787657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c1a8a-1a63-4708-bb9d-b2681592fe75 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:35.803150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 179/341] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:35.810023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00cb6-1345-419f-998c-ef177faba818 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.815331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00cb6-1345-419f-998c-ef177faba818 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.815331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00cb6-1345-419f-998c-ef177faba818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.848338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8bb00cb6-1345-419f-998c-ef177faba818 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:35.849624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8bb00cb6-1345-419f-998c-ef177faba818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:35.866939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:35.878617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219554d-00fb-422f-8994-9f101cdb97ac None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:35.915963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000297c9-c4ee-47f9-8b42-0b7c09220997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:35.917061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000297c9-c4ee-47f9-8b42-0b7c09220997 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.917553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000297c9-c4ee-47f9-8b42-0b7c09220997 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.918016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000297c9-c4ee-47f9-8b42-0b7c09220997 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.929287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000297c9-c4ee-47f9-8b42-0b7c09220997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-000297c9-c4ee-47f9-8b42-0b7c09220997 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:35.940415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:35.941335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:32:35.942095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:35.945062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 163/342] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:35.946079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000297c9-c4ee-47f9-8b42-0b7c09220997 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.033487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 180/343] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:36.034286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb00cb6-1345-419f-998c-ef177faba818 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.054613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000297c9-c4ee-47f9-8b42-0b7c09220997 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:36.091543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6096a8f-cf06-456d-8c48-c7ea3bf8b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:36.092468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6096a8f-cf06-456d-8c48-c7ea3bf8b303 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.092963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6096a8f-cf06-456d-8c48-c7ea3bf8b303 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.093395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6096a8f-cf06-456d-8c48-c7ea3bf8b303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.124818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:36.251478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:36.255877 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb00cb6-1345-419f-998c-ef177faba818 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:36.264396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6096a8f-cf06-456d-8c48-c7ea3bf8b303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:36.266459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6096a8f-cf06-456d-8c48-c7ea3bf8b303 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:36.299815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:36.300070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.300338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.300615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.301482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] RBAC: Adding query filter params (name=reader) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:36.301793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:36.303890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:36.313684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 181/344] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:36.314773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.398357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 164/345] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:36.405858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febfb44-25bf-4176-ae7e-e4574c844456 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.405858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febfb44-25bf-4176-ae7e-e4574c844456 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.405858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febfb44-25bf-4176-ae7e-e4574c844456 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.405858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 165/346] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:36.408264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febfb44-25bf-4176-ae7e-e4574c844456 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.444527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fe2de-598d-4c07-8938-b1a78b5be61f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:36.474459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:36.484609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fd4b8-fffe-4a70-bf92-6be6ea061164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:36.485715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd4b8-fffe-4a70-bf92-6be6ea061164 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.486209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd4b8-fffe-4a70-bf92-6be6ea061164 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.486682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fd4b8-fffe-4a70-bf92-6be6ea061164 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.498638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683fd4b8-fffe-4a70-bf92-6be6ea061164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-683fd4b8-fffe-4a70-bf92-6be6ea061164 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:36.507882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:36.508757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:36.509537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:36.511912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 182/347] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:36.512909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fd4b8-fffe-4a70-bf92-6be6ea061164 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.529111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febfb44-25bf-4176-ae7e-e4574c844456 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:36.545938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:36.567818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45654df1-81c5-4822-9d4d-2115b353787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:36.568723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654df1-81c5-4822-9d4d-2115b353787d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.569016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654df1-81c5-4822-9d4d-2115b353787d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.569347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654df1-81c5-4822-9d4d-2115b353787d None admin] PATH_INFO: `/v3/users/9c439831936842d3a5ba93162dd84eb9` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.570567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654df1-81c5-4822-9d4d-2115b353787d None admin] RBAC: Authorizing `identity:get_user(user_id=9c439831936842d3a5ba93162dd84eb9)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:36.572120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654df1-81c5-4822-9d4d-2115b353787d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:36.573776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 166/348] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:32:36 2022] GET /identity/v3/users/9c439831936842d3a5ba93162dd84eb9 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:36.574990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc95bc-a831-4091-86eb-c359c20733cf None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.575283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc95bc-a831-4091-86eb-c359c20733cf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.575620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc95bc-a831-4091-86eb-c359c20733cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.596447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1bfc95bc-a831-4091-86eb-c359c20733cf None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:36.597722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1bfc95bc-a831-4091-86eb-c359c20733cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:36.634535 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fd4b8-fffe-4a70-bf92-6be6ea061164 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:36.671414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 167/349] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:36.671617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ccc15-3d3b-4569-b3c4-12e3e4a580e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:36.672343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc95bc-a831-4091-86eb-c359c20733cf None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.672933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ccc15-3d3b-4569-b3c4-12e3e4a580e3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.673424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ccc15-3d3b-4569-b3c4-12e3e4a580e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.673908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ccc15-3d3b-4569-b3c4-12e3e4a580e3 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.675361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ccc15-3d3b-4569-b3c4-12e3e4a580e3 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:36.677034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ccc15-3d3b-4569-b3c4-12e3e4a580e3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:36.679028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 183/350] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:36 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:36.680013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ccc15-3d3b-4569-b3c4-12e3e4a580e3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.730690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:36.783441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:36.785157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc95bc-a831-4091-86eb-c359c20733cf None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:36.798404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ccc15-3d3b-4569-b3c4-12e3e4a580e3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:36.823895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:36.824611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.824956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.825254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.826006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:36.826314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:36.827760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:36.839657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 168/351] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:36.840546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.841331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:36.842181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:36.842664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:36.843111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:36.844410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] RBAC: Adding query filter params (role.id=3555f68f0fca46018de9022f06dcdc90, user.id=9c439831936842d3a5ba93162dd84eb9, scope.system=all) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:36.845758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:36.847445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:36.863734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 184/352] 10.222.0.98 () {56 vars in 1308 bytes} [Fri Jul 15 15:32:36 2022] GET /identity/v3/role_assignments?user.id=9c439831936842d3a5ba93162dd84eb9&scope.system=all&role.id=3555f68f0fca46018de9022f06dcdc90 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:36.864412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:36.969519 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24efa9b-4ae2-49c3-a1bf-0451a0724c66 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:36.984584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:37.008771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d103c-af4a-4885-abcb-b7e6ad1b2fa1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:37.014927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0023a26-c24e-4903-9360-63a07f352d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:37.016358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023a26-c24e-4903-9360-63a07f352d75 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.016854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023a26-c24e-4903-9360-63a07f352d75 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.017336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0023a26-c24e-4903-9360-63a07f352d75 None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.026349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0023a26-c24e-4903-9360-63a07f352d75 None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:37.028311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0023a26-c24e-4903-9360-63a07f352d75 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:37.030873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 169/353] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:36 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:37.032267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0023a26-c24e-4903-9360-63a07f352d75 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:37.059011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97c55f-8ca0-46fd-bfef-bcc74e87e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:37.059885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97c55f-8ca0-46fd-bfef-bcc74e87e21f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.060177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97c55f-8ca0-46fd-bfef-bcc74e87e21f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.060461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97c55f-8ca0-46fd-bfef-bcc74e87e21f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.075924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97c55f-8ca0-46fd-bfef-bcc74e87e21f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ed97c55f-8ca0-46fd-bfef-bcc74e87e21f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:37.086043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:37.086972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:37.087837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 185/354] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:37.088496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97c55f-8ca0-46fd-bfef-bcc74e87e21f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:37.108888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:37.172366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0023a26-c24e-4903-9360-63a07f352d75 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:37.216732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c3249-fad9-4c79-a95c-5107c222f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:37.217909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c3249-fad9-4c79-a95c-5107c222f33a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.218704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c3249-fad9-4c79-a95c-5107c222f33a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.219493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c3249-fad9-4c79-a95c-5107c222f33a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.224996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97c55f-8ca0-46fd-bfef-bcc74e87e21f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:37.272307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:37.275721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.276026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.276318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.277186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] RBAC: Adding query filter params (group.id=d71a2f7b114c4155badd9ac70c64b325, role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=b7fddf3939de4bb99741de8e3307565f) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:37.277513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:37.279005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:37.286701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 186/355] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:37 2022] GET /identity/v3/role_assignments?group.id=d71a2f7b114c4155badd9ac70c64b325&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:37.287776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:37.289083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:37.322434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c3249-fad9-4c79-a95c-5107c222f33a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:37.324478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c3249-fad9-4c79-a95c-5107c222f33a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:37.436021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:37.443674 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 170/356] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:37.449101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c3249-fad9-4c79-a95c-5107c222f33a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:37.529415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7048c7-8ed6-4342-b17c-95119f9e1855 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:37.561189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:37.575114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60db5df-fe25-481a-95a7-f8857a528d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:37.575905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db5df-fe25-481a-95a7-f8857a528d12 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.576195 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db5df-fe25-481a-95a7-f8857a528d12 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.580186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60db5df-fe25-481a-95a7-f8857a528d12 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.580186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60db5df-fe25-481a-95a7-f8857a528d12 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:37.580186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60db5df-fe25-481a-95a7-f8857a528d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:37.580186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60db5df-fe25-481a-95a7-f8857a528d12 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:37.588841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 187/357] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:37.590154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115bc6-9baa-49f4-a721-ad86e4466495 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.590448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115bc6-9baa-49f4-a721-ad86e4466495 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.590730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115bc6-9baa-49f4-a721-ad86e4466495 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.592526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 188/358] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:37.594690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115bc6-9baa-49f4-a721-ad86e4466495 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:37.596067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c3249-fad9-4c79-a95c-5107c222f33a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:37.648949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912b2fc-c445-4b9b-ae95-3c53bca62e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:37.650084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b2fc-c445-4b9b-ae95-3c53bca62e62 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.650373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b2fc-c445-4b9b-ae95-3c53bca62e62 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.650654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912b2fc-c445-4b9b-ae95-3c53bca62e62 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.652675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912b2fc-c445-4b9b-ae95-3c53bca62e62 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:37.654159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912b2fc-c445-4b9b-ae95-3c53bca62e62 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:37.659444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 171/359] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:37 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:37.659444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52f12-72a9-4341-be2b-5e54e3abd374 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.659444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52f12-72a9-4341-be2b-5e54e3abd374 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.659444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52f12-72a9-4341-be2b-5e54e3abd374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.680025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-46e52f12-72a9-4341-be2b-5e54e3abd374 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:37.681369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-46e52f12-72a9-4341-be2b-5e54e3abd374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:37.744093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115bc6-9baa-49f4-a721-ad86e4466495 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:37.773989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 172/360] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:37.775862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228508a5-421f-4143-a7ca-d8a4945362f9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.776247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228508a5-421f-4143-a7ca-d8a4945362f9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.776673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228508a5-421f-4143-a7ca-d8a4945362f9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.779165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 173/361] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:37.779771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228508a5-421f-4143-a7ca-d8a4945362f9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:37.783986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee770d3-718e-4eef-8108-ef3f1bbd9d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:37.784751 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee770d3-718e-4eef-8108-ef3f1bbd9d07 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.785110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee770d3-718e-4eef-8108-ef3f1bbd9d07 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.785466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee770d3-718e-4eef-8108-ef3f1bbd9d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.788949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:37.880050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee770d3-718e-4eef-8108-ef3f1bbd9d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:37.881858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee770d3-718e-4eef-8108-ef3f1bbd9d07 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:37.887708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:37.917369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228508a5-421f-4143-a7ca-d8a4945362f9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:37.962789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d7e52-e81d-4940-a122-2e469b30fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:37.963757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d7e52-e81d-4940-a122-2e469b30fe27 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.964080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d7e52-e81d-4940-a122-2e469b30fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.964443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d7e52-e81d-4940-a122-2e469b30fe27 None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.966377 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d7e52-e81d-4940-a122-2e469b30fe27 None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:37.967960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d7e52-e81d-4940-a122-2e469b30fe27 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:37.969611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 174/362] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:37 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:37.970884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47cc9-719e-4f8d-b4f7-6a88074ea86a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:37.971225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47cc9-719e-4f8d-b4f7-6a88074ea86a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:37.971566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47cc9-719e-4f8d-b4f7-6a88074ea86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:37.988928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7fc47cc9-719e-4f8d-b4f7-6a88074ea86a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:37.990661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7fc47cc9-719e-4f8d-b4f7-6a88074ea86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:38.011537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 189/363] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:38.013127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee770d3-718e-4eef-8108-ef3f1bbd9d07 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.067159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:38.078879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 175/364] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:38.080068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc47cc9-719e-4f8d-b4f7-6a88074ea86a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.143966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee770d3-718e-4eef-8108-ef3f1bbd9d07 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.164581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:38.192603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9474fab-bc51-4add-a388-cb39a41a06b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.194480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9474fab-bc51-4add-a388-cb39a41a06b4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.195546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9474fab-bc51-4add-a388-cb39a41a06b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.196127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9474fab-bc51-4add-a388-cb39a41a06b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.208147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9474fab-bc51-4add-a388-cb39a41a06b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:38.209502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc47cc9-719e-4f8d-b4f7-6a88074ea86a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a9474fab-bc51-4add-a388-cb39a41a06b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:38.216861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:38.217754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:38.218523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 190/365] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:38.219173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9474fab-bc51-4add-a388-cb39a41a06b4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.260061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.260419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.260604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.260876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.261734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] RBAC: Adding query filter params (group.id=d71a2f7b114c4155badd9ac70c64b325, role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=b7fddf3939de4bb99741de8e3307565f) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:38.262044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:38.264313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:38.279674 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 176/366] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/role_assignments?group.id=d71a2f7b114c4155badd9ac70c64b325&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:38.281180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.357092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:38.367871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9474fab-bc51-4add-a388-cb39a41a06b4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.422866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764b5f-b509-419c-aebb-36ca6f73e3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.423945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764b5f-b509-419c-aebb-36ca6f73e3b1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.424331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764b5f-b509-419c-aebb-36ca6f73e3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.424710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764b5f-b509-419c-aebb-36ca6f73e3b1 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.426051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd764b5f-b509-419c-aebb-36ca6f73e3b1 None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:38.428264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd764b5f-b509-419c-aebb-36ca6f73e3b1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:38.429331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9f2f6-7d9c-4acb-874c-0fa784b4b280 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.430495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 191/367] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:38.431217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764b5f-b509-419c-aebb-36ca6f73e3b1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.481284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce299f-e096-4579-890c-3e87f9bca4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.482153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce299f-e096-4579-890c-3e87f9bca4f8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.482435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce299f-e096-4579-890c-3e87f9bca4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.482711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce299f-e096-4579-890c-3e87f9bca4f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.486348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:38.495984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce299f-e096-4579-890c-3e87f9bca4f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b8ce299f-e096-4579-890c-3e87f9bca4f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:38.504510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:38.505631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:38.506580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 177/368] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:38.507277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce299f-e096-4579-890c-3e87f9bca4f8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.563084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764b5f-b509-419c-aebb-36ca6f73e3b1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.609385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.610333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.610783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.611223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.612198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:38.612666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:38.614248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:38.623718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 192/369] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:38.624752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.631349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce299f-e096-4579-890c-3e87f9bca4f8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.656543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:38.671378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7565-4405-4e95-b0b6-65f80a6068d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.672481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7565-4405-4e95-b0b6-65f80a6068d6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.672915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7565-4405-4e95-b0b6-65f80a6068d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.675774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7565-4405-4e95-b0b6-65f80a6068d6 None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.675774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e7565-4405-4e95-b0b6-65f80a6068d6 None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:38.678847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e7565-4405-4e95-b0b6-65f80a6068d6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:38.682697 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 178/370] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:38.683800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7565-4405-4e95-b0b6-65f80a6068d6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.761138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842aee56-348d-4b5f-8263-c61cc322753f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.788006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:38.799349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.800179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.800591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.800973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.801829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:38.802241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:38.803979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:38.813018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 193/371] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:38.814182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:38.834512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7565-4405-4e95-b0b6-65f80a6068d6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.874124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ef807-df66-4aa7-8e72-df0619e5ae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.875148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef807-df66-4aa7-8e72-df0619e5ae60 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.875696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef807-df66-4aa7-8e72-df0619e5ae60 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.876362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ef807-df66-4aa7-8e72-df0619e5ae60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.906836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:38.923271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe69db-ad30-4a72-bfa2-ebc99a75144b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:38.959497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ef807-df66-4aa7-8e72-df0619e5ae60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:38.960507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:38.961175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:38.961465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:38.961749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:38.962262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ef807-df66-4aa7-8e72-df0619e5ae60 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:38.962587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] RBAC: Adding query filter params (group.id=d71a2f7b114c4155badd9ac70c64b325, role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=366bead620044a59b9ae769237db8568) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:38.962907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:38.964397 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:38.971239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 194/372] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/role_assignments?group.id=d71a2f7b114c4155badd9ac70c64b325&scope.project.id=366bead620044a59b9ae769237db8568&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:38.972124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:39.044097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:39.086685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 179/373] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:39.093263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ef807-df66-4aa7-8e72-df0619e5ae60 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:39.144450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786284f2-b99c-462a-9944-cfdefd03c598 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:39.216955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351b6df-4d42-4612-807a-2f12c517c75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:39.217642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351b6df-4d42-4612-807a-2f12c517c75f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:39.217929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351b6df-4d42-4612-807a-2f12c517c75f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:39.218210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f351b6df-4d42-4612-807a-2f12c517c75f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:39.218686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ef807-df66-4aa7-8e72-df0619e5ae60 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:39.266136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:39.269194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f4488-df6d-40f3-bd0c-885e83a19733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:39.270481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f4488-df6d-40f3-bd0c-885e83a19733 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:39.270996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f4488-df6d-40f3-bd0c-885e83a19733 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:39.272093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f4488-df6d-40f3-bd0c-885e83a19733 None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:39.276114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f4488-df6d-40f3-bd0c-885e83a19733 None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:39.277974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f4488-df6d-40f3-bd0c-885e83a19733 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:39.280288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 180/374] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:39 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:39.290101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f4488-df6d-40f3-bd0c-885e83a19733 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:39.339797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:39.354544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351b6df-4d42-4612-807a-2f12c517c75f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:39.356181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f351b6df-4d42-4612-807a-2f12c517c75f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:39.451710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f4488-df6d-40f3-bd0c-885e83a19733 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:39.498225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058d301-6f2e-4857-a355-ad30a8687286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:39.504720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058d301-6f2e-4857-a355-ad30a8687286 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:39.505234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058d301-6f2e-4857-a355-ad30a8687286 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:39.506296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058d301-6f2e-4857-a355-ad30a8687286 None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:39.508105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058d301-6f2e-4857-a355-ad30a8687286 None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:39.509932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058d301-6f2e-4857-a355-ad30a8687286 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:39.512546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 181/375] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:39 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:39.518916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058d301-6f2e-4857-a355-ad30a8687286 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:39.611260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 195/376] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:39.614050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351b6df-4d42-4612-807a-2f12c517c75f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:39.689909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:39.693334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058d301-6f2e-4857-a355-ad30a8687286 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:39.723690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:39.736799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:39.737694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:39.738076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:39.738429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:39.739696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] RBAC: Adding query filter params (group.id=d71a2f7b114c4155badd9ac70c64b325, role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=366bead620044a59b9ae769237db8568) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:39.740738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:39.742495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:39.749306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 182/377] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:39 2022] GET /identity/v3/role_assignments?group.id=d71a2f7b114c4155badd9ac70c64b325&scope.project.id=366bead620044a59b9ae769237db8568&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:39.804135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f351b6df-4d42-4612-807a-2f12c517c75f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:39.847707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b30ab-00c8-49ff-8036-4f1d29440015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:39.848585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b30ab-00c8-49ff-8036-4f1d29440015 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:39.848868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b30ab-00c8-49ff-8036-4f1d29440015 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:39.849142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b30ab-00c8-49ff-8036-4f1d29440015 None admin] PATH_INFO: `/v3/projects/c42afd51dd0d44aba511a26dccae48ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:39.850434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b30ab-00c8-49ff-8036-4f1d29440015 None admin] RBAC: Authorizing `identity:get_project(project_id=c42afd51dd0d44aba511a26dccae48ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:39.852099 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52b30ab-00c8-49ff-8036-4f1d29440015 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:39.854232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 196/378] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:39 2022] GET /identity/v3/projects/c42afd51dd0d44aba511a26dccae48ce => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:39.859616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:39.869334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49aa26-6c33-4aa2-a3e8-784d0ad7acc5 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:39.869644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49aa26-6c33-4aa2-a3e8-784d0ad7acc5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:39.869809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49aa26-6c33-4aa2-a3e8-784d0ad7acc5 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:39.871436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 197/379] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:39.877945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902264-a1cd-4c5f-a0c7-49768a2786ea None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:39.878587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902264-a1cd-4c5f-a0c7-49768a2786ea None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:39.879068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa902264-a1cd-4c5f-a0c7-49768a2786ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:39.897771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fa902264-a1cd-4c5f-a0c7-49768a2786ea None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:39.899704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fa902264-a1cd-4c5f-a0c7-49768a2786ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:40.004499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 198/380] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:40.030920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d46af-3e79-495e-b260-3c3a31dd743a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:40.034115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa902264-a1cd-4c5f-a0c7-49768a2786ea None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:40.074993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af7d02-5a59-431a-aea5-846dddd206e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:40.075893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af7d02-5a59-431a-aea5-846dddd206e5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:40.076184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af7d02-5a59-431a-aea5-846dddd206e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:40.076464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af7d02-5a59-431a-aea5-846dddd206e5 None admin] PATH_INFO: `/v3/groups/8b3041d67d4e4fa3853ab2207d2d695d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:40.088034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af7d02-5a59-431a-aea5-846dddd206e5 None admin] RBAC: Authorizing `identity:get_group(group_id=8b3041d67d4e4fa3853ab2207d2d695d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:40.089528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af7d02-5a59-431a-aea5-846dddd206e5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:40.092503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:40.092627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 183/381] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:39 2022] GET /identity/v3/groups/8b3041d67d4e4fa3853ab2207d2d695d => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:40.098396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af7d02-5a59-431a-aea5-846dddd206e5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:40.131788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:40.169724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa902264-a1cd-4c5f-a0c7-49768a2786ea None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:40.210625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af7d02-5a59-431a-aea5-846dddd206e5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:40.223686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69642-a296-49aa-91da-52c9ee66c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:40.225347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69642-a296-49aa-91da-52c9ee66c373 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:40.225845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69642-a296-49aa-91da-52c9ee66c373 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:40.226294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d69642-a296-49aa-91da-52c9ee66c373 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:40.249604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d69642-a296-49aa-91da-52c9ee66c373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a7d69642-a296-49aa-91da-52c9ee66c373 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:40.257649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:40.258538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:40.259336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:40.260148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 199/382] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:40.266561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69642-a296-49aa-91da-52c9ee66c373 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:40.279919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:40.280580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:40.280862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:40.281135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:40.281970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] RBAC: Adding query filter params (group.id=8b3041d67d4e4fa3853ab2207d2d695d, role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=c42afd51dd0d44aba511a26dccae48ce) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:40.282280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:40.284403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:40.295313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 184/383] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:40 2022] GET /identity/v3/role_assignments?group.id=8b3041d67d4e4fa3853ab2207d2d695d&scope.project.id=c42afd51dd0d44aba511a26dccae48ce&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:40.431058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d69642-a296-49aa-91da-52c9ee66c373 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:40.444945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:40.477007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:40.477690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:40.477971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:40.478249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:40.479100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:40.479442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:40.480859 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:40.494055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 200/384] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:40.500315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:40.625863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:40.635835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c0713-563b-46a1-b1e7-f25f2dec1084 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:40.672143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0f8d9-5ae1-4af9-ab82-633d8e8f6779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:40.672992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0f8d9-5ae1-4af9-ab82-633d8e8f6779 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:40.673270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0f8d9-5ae1-4af9-ab82-633d8e8f6779 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:40.673544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0f8d9-5ae1-4af9-ab82-633d8e8f6779 None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:40.674772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0f8d9-5ae1-4af9-ab82-633d8e8f6779 None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:40.676245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0f8d9-5ae1-4af9-ab82-633d8e8f6779 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:40.678210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 185/385] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:40 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:40.683770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:40.848454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29520c-c2de-416b-b6a7-cc1cf13a3942 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:40.892048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a0a3e-8ef1-45b9-bac2-83dfc0ce93e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:40.893285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0a3e-8ef1-45b9-bac2-83dfc0ce93e0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:40.893793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0a3e-8ef1-45b9-bac2-83dfc0ce93e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:40.894291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0a3e-8ef1-45b9-bac2-83dfc0ce93e0 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:40.895756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a0a3e-8ef1-45b9-bac2-83dfc0ce93e0 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:40.897983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a0a3e-8ef1-45b9-bac2-83dfc0ce93e0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:40.900546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 201/386] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:40 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:40.906518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0f8d9-5ae1-4af9-ab82-633d8e8f6779 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:41.070305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0f8d9-5ae1-4af9-ab82-633d8e8f6779 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:41.114779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9cac4-8eb3-4899-ac01-f4ce3b4e3474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:41.116256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9cac4-8eb3-4899-ac01-f4ce3b4e3474 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:41.116539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9cac4-8eb3-4899-ac01-f4ce3b4e3474 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:41.116812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9cac4-8eb3-4899-ac01-f4ce3b4e3474 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f/groups/d71a2f7b114c4155badd9ac70c64b325/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:41.119041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9cac4-8eb3-4899-ac01-f4ce3b4e3474 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7fddf3939de4bb99741de8e3307565f, group_id=d71a2f7b114c4155badd9ac70c64b325, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:41.120532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9cac4-8eb3-4899-ac01-f4ce3b4e3474 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:41.132518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0159e0-fac4-4b1a-89c1-c7b5925a6c35 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:41.132815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0159e0-fac4-4b1a-89c1-c7b5925a6c35 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:41.133074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0159e0-fac4-4b1a-89c1-c7b5925a6c35 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:41.134715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 202/387] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:41.140211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e2c94-0494-4035-a719-0a6a92f3d043 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:41.140504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e2c94-0494-4035-a719-0a6a92f3d043 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:41.140787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e2c94-0494-4035-a719-0a6a92f3d043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:41.166243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b82e2c94-0494-4035-a719-0a6a92f3d043 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:41.167471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b82e2c94-0494-4035-a719-0a6a92f3d043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:41.212542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 186/388] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Jul 15 15:32:40 2022] PUT /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f/groups/d71a2f7b114c4155badd9ac70c64b325/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:41.217124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:41.268799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 203/389] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:41.300281 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9cac4-8eb3-4899-ac01-f4ce3b4e3474 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:41.336848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:41.432168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9cac4-8eb3-4899-ac01-f4ce3b4e3474 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:41.467246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016b32b-136a-4057-af32-49cefe82340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:41.468081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b32b-136a-4057-af32-49cefe82340c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:41.468364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b32b-136a-4057-af32-49cefe82340c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:41.468639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b32b-136a-4057-af32-49cefe82340c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:41.479023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016b32b-136a-4057-af32-49cefe82340c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c016b32b-136a-4057-af32-49cefe82340c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:41.487013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:41.487906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:41.488665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 187/390] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:41.496318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e2c94-0494-4035-a719-0a6a92f3d043 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:41.633418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e2c94-0494-4035-a719-0a6a92f3d043 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:41.645249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5549f4-c777-488c-ae43-ccbdd1b5ec92 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:41.645730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5549f4-c777-488c-ae43-ccbdd1b5ec92 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:41.645830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5549f4-c777-488c-ae43-ccbdd1b5ec92 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:41.647567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 188/391] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:41.654323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a2e1-b6bc-44a1-a043-1546618fdcf8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:41.654582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a2e1-b6bc-44a1-a043-1546618fdcf8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:41.654864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a2e1-b6bc-44a1-a043-1546618fdcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:41.688868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-32c8a2e1-b6bc-44a1-a043-1546618fdcf8 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:41.690097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-32c8a2e1-b6bc-44a1-a043-1546618fdcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:41.696584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:41.697275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:41.697566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:41.697849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:41.698605 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:41.698917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:41.700344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:41.716654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 204/392] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:41.722606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:41.790686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:41.828610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 189/393] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:41.852300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a2e1-b6bc-44a1-a043-1546618fdcf8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:41.876679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1fb2-0720-482a-bf57-745605da5736 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:41.881397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:41.935490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6949b-f526-49a5-962a-2db6848554c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:41.936345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6949b-f526-49a5-962a-2db6848554c1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:41.936640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6949b-f526-49a5-962a-2db6848554c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:41.936920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6949b-f526-49a5-962a-2db6848554c1 None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:41.939351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6949b-f526-49a5-962a-2db6848554c1 None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:41.940856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6949b-f526-49a5-962a-2db6848554c1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:41.942888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 205/394] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:41 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:41.948230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6949b-f526-49a5-962a-2db6848554c1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:42.086044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a2e1-b6bc-44a1-a043-1546618fdcf8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:42.098681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:42.107432 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6949b-f526-49a5-962a-2db6848554c1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:42.140305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696ab54-6012-4fe7-b2f2-c176357e5e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:42.142177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ab54-6012-4fe7-b2f2-c176357e5e1e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.142843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ab54-6012-4fe7-b2f2-c176357e5e1e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.143575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696ab54-6012-4fe7-b2f2-c176357e5e1e None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.143790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff452-9fce-461c-a968-517f1cbbaf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:42.144771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff452-9fce-461c-a968-517f1cbbaf99 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.145132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696ab54-6012-4fe7-b2f2-c176357e5e1e None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:42.145353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff452-9fce-461c-a968-517f1cbbaf99 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.145780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff452-9fce-461c-a968-517f1cbbaf99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.151319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696ab54-6012-4fe7-b2f2-c176357e5e1e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:42.151319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 206/395] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:41 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:42.156941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696ab54-6012-4fe7-b2f2-c176357e5e1e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:42.165591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ff452-9fce-461c-a968-517f1cbbaf99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-663ff452-9fce-461c-a968-517f1cbbaf99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:42.181856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:42.182762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:42.183636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:42.186631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 190/396] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:42.192152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff452-9fce-461c-a968-517f1cbbaf99 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:42.267143 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696ab54-6012-4fe7-b2f2-c176357e5e1e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:42.306106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bda4f-cc1d-40c8-97b3-2c113f53dea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:42.307100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bda4f-cc1d-40c8-97b3-2c113f53dea7 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.307437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bda4f-cc1d-40c8-97b3-2c113f53dea7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.307725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bda4f-cc1d-40c8-97b3-2c113f53dea7 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f/groups/d71a2f7b114c4155badd9ac70c64b325/roles/9a22f6f23fb64da488d982c17ed565ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.309993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bda4f-cc1d-40c8-97b3-2c113f53dea7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7fddf3939de4bb99741de8e3307565f, group_id=d71a2f7b114c4155badd9ac70c64b325, role_id=9a22f6f23fb64da488d982c17ed565ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:42.311498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bda4f-cc1d-40c8-97b3-2c113f53dea7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:42.315813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:42.352960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff452-9fce-461c-a968-517f1cbbaf99 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:42.388483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 207/397] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Jul 15 15:32:42 2022] PUT /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f/groups/d71a2f7b114c4155badd9ac70c64b325/roles/9a22f6f23fb64da488d982c17ed565ce => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:32:42.401631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea305cc-bdde-4732-9579-e71ae4c32784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:42.406908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea305cc-bdde-4732-9579-e71ae4c32784 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.406908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea305cc-bdde-4732-9579-e71ae4c32784 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.406908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea305cc-bdde-4732-9579-e71ae4c32784 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.406908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea305cc-bdde-4732-9579-e71ae4c32784 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:42.406908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea305cc-bdde-4732-9579-e71ae4c32784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:42.406908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea305cc-bdde-4732-9579-e71ae4c32784 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:42.421723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 191/398] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:42.427474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bda4f-cc1d-40c8-97b3-2c113f53dea7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:42.471936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:42.590699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3a693-9f08-4cf1-ad1a-82aa8f6606d7 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.590892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3a693-9f08-4cf1-ad1a-82aa8f6606d7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.591153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3a693-9f08-4cf1-ad1a-82aa8f6606d7 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.592325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bda4f-cc1d-40c8-97b3-2c113f53dea7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:42.592655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 192/399] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:42.601294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad2a09-dbc2-43ab-9d1d-4c1a751b743f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.601579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad2a09-dbc2-43ab-9d1d-4c1a751b743f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.601852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad2a09-dbc2-43ab-9d1d-4c1a751b743f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.626372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9aad2a09-dbc2-43ab-9d1d-4c1a751b743f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:42.627701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9aad2a09-dbc2-43ab-9d1d-4c1a751b743f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:42.640885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612e4ca-73e2-47d4-891d-c42170c31389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:42.646134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e4ca-73e2-47d4-891d-c42170c31389 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.646134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e4ca-73e2-47d4-891d-c42170c31389 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.646134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612e4ca-73e2-47d4-891d-c42170c31389 None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.646134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612e4ca-73e2-47d4-891d-c42170c31389 None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:42.646134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612e4ca-73e2-47d4-891d-c42170c31389 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:42.647875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 208/400] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:42 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:42.653514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612e4ca-73e2-47d4-891d-c42170c31389 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:42.706749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 193/401] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:42.730061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad2a09-dbc2-43ab-9d1d-4c1a751b743f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:42.791643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:42.806484 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612e4ca-73e2-47d4-891d-c42170c31389 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:42.811357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:42.867532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0db29-3142-47e4-ad16-48ea983529ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:42.868680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0db29-3142-47e4-ad16-48ea983529ef None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.869117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0db29-3142-47e4-ad16-48ea983529ef None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.869559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0db29-3142-47e4-ad16-48ea983529ef None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.870908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0db29-3142-47e4-ad16-48ea983529ef None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:42.873656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0db29-3142-47e4-ad16-48ea983529ef None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:42.877359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 209/402] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:42 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:42.883848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0db29-3142-47e4-ad16-48ea983529ef None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:42.886576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad2a09-dbc2-43ab-9d1d-4c1a751b743f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:42.923907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8b387-5231-4568-9164-2b0e56caf916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:42.925136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8b387-5231-4568-9164-2b0e56caf916 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.925650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8b387-5231-4568-9164-2b0e56caf916 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.926128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8b387-5231-4568-9164-2b0e56caf916 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.938209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8b387-5231-4568-9164-2b0e56caf916 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-48d8b387-5231-4568-9164-2b0e56caf916 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:42.947242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:42.948139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:42.949007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:42.951790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 194/403] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:42.953348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5a74f-6a41-4ac4-889c-9b36bd834d6f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:42.953866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5a74f-6a41-4ac4-889c-9b36bd834d6f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:42.954345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5a74f-6a41-4ac4-889c-9b36bd834d6f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:42.956319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 195/404] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:42.957272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5a74f-6a41-4ac4-889c-9b36bd834d6f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:43.004904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0db29-3142-47e4-ad16-48ea983529ef None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:43.042112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb1319-af54-49ac-9ae7-ade7e331258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:43.043381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb1319-af54-49ac-9ae7-ade7e331258e None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.043493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb1319-af54-49ac-9ae7-ade7e331258e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.043713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb1319-af54-49ac-9ae7-ade7e331258e None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568/groups/d71a2f7b114c4155badd9ac70c64b325/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.045708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb1319-af54-49ac-9ae7-ade7e331258e None admin] RBAC: Authorizing `identity:create_grant(project_id=366bead620044a59b9ae769237db8568, group_id=d71a2f7b114c4155badd9ac70c64b325, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:43.047283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb1319-af54-49ac-9ae7-ade7e331258e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:43.071876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5a74f-6a41-4ac4-889c-9b36bd834d6f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:43.096126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:43.108238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c7ce1-8f1b-4517-80ba-c614c4d046aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:43.108909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7ce1-8f1b-4517-80ba-c614c4d046aa None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.109202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7ce1-8f1b-4517-80ba-c614c4d046aa None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.109483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c7ce1-8f1b-4517-80ba-c614c4d046aa None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.110241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c7ce1-8f1b-4517-80ba-c614c4d046aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:43.110536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c7ce1-8f1b-4517-80ba-c614c4d046aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:43.111957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c7ce1-8f1b-4517-80ba-c614c4d046aa None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:43.122201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 196/405] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:43.123831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd2820-95f4-4b3e-8609-f6833d24973b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.124139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd2820-95f4-4b3e-8609-f6833d24973b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.124492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd2820-95f4-4b3e-8609-f6833d24973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.127474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 210/406] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Jul 15 15:32:42 2022] PUT /identity/v3/projects/366bead620044a59b9ae769237db8568/groups/d71a2f7b114c4155badd9ac70c64b325/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:32:43.133846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb1319-af54-49ac-9ae7-ade7e331258e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:43.144107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-68bd2820-95f4-4b3e-8609-f6833d24973b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:43.145791 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-68bd2820-95f4-4b3e-8609-f6833d24973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:43.163209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:43.266536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 197/407] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:43.289037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb1319-af54-49ac-9ae7-ade7e331258e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:43.303331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd2820-95f4-4b3e-8609-f6833d24973b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:43.350986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e841d1-6b3b-423a-80c1-0cbd7ca8be0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:43.351948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e841d1-6b3b-423a-80c1-0cbd7ca8be0c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.352236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e841d1-6b3b-423a-80c1-0cbd7ca8be0c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.352515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e841d1-6b3b-423a-80c1-0cbd7ca8be0c None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.354349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e841d1-6b3b-423a-80c1-0cbd7ca8be0c None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:43.355929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e841d1-6b3b-423a-80c1-0cbd7ca8be0c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:43.357915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 211/408] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:43 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:43.363070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e841d1-6b3b-423a-80c1-0cbd7ca8be0c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:43.437760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd2820-95f4-4b3e-8609-f6833d24973b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:43.455780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:43.477267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78ed4b-5f9f-415c-8c42-e998732ad1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:43.478388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78ed4b-5f9f-415c-8c42-e998732ad1ca None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.478838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78ed4b-5f9f-415c-8c42-e998732ad1ca None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.481368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78ed4b-5f9f-415c-8c42-e998732ad1ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.491021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78ed4b-5f9f-415c-8c42-e998732ad1ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-6e78ed4b-5f9f-415c-8c42-e998732ad1ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:43.498993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:43.499884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:43.500656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:43.503385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 198/409] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:32:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:43.507692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:43.510645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78ed4b-5f9f-415c-8c42-e998732ad1ca None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:43.533048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e841d1-6b3b-423a-80c1-0cbd7ca8be0c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:43.591466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8e876-0690-4279-a519-43ef535014ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:43.592342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e876-0690-4279-a519-43ef535014ba None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.592632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e876-0690-4279-a519-43ef535014ba None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.592912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e876-0690-4279-a519-43ef535014ba None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.594498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8e876-0690-4279-a519-43ef535014ba None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:43.596147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8e876-0690-4279-a519-43ef535014ba None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:43.598469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 212/410] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:43 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:43.604621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8e876-0690-4279-a519-43ef535014ba None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:43.720147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8e876-0690-4279-a519-43ef535014ba None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:43.761935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78ed4b-5f9f-415c-8c42-e998732ad1ca None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:43.781543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0713af-7d97-4470-9446-69d558fa2b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:43.782839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0713af-7d97-4470-9446-69d558fa2b6b None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.783183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0713af-7d97-4470-9446-69d558fa2b6b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.783434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0713af-7d97-4470-9446-69d558fa2b6b None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568/groups/d71a2f7b114c4155badd9ac70c64b325/roles/9a22f6f23fb64da488d982c17ed565ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.787970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0713af-7d97-4470-9446-69d558fa2b6b None admin] RBAC: Authorizing `identity:create_grant(project_id=366bead620044a59b9ae769237db8568, group_id=d71a2f7b114c4155badd9ac70c64b325, role_id=9a22f6f23fb64da488d982c17ed565ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:43.789493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0713af-7d97-4470-9446-69d558fa2b6b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:43.816001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:43.819456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.819456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.819456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.819456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:43.819456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:43.820294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:43.829369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 199/411] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:43.834726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:43.873308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:43.880693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 213/412] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Jul 15 15:32:43 2022] PUT /identity/v3/projects/366bead620044a59b9ae769237db8568/groups/d71a2f7b114c4155badd9ac70c64b325/roles/9a22f6f23fb64da488d982c17ed565ce => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:32:43.943325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e38fa-d666-42c5-bbe2-27615d9a5cdc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:43.978928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac87e4-f8a9-43f9-ad91-152ef8c4b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:43.979847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac87e4-f8a9-43f9-ad91-152ef8c4b8fc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:43.980131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac87e4-f8a9-43f9-ad91-152ef8c4b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:43.980407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac87e4-f8a9-43f9-ad91-152ef8c4b8fc None admin] PATH_INFO: `/v3/groups/8b3041d67d4e4fa3853ab2207d2d695d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:43.981773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac87e4-f8a9-43f9-ad91-152ef8c4b8fc None admin] RBAC: Authorizing `identity:get_group(group_id=8b3041d67d4e4fa3853ab2207d2d695d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:43.983330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac87e4-f8a9-43f9-ad91-152ef8c4b8fc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:43.985422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 200/413] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:43 2022] GET /identity/v3/groups/8b3041d67d4e4fa3853ab2207d2d695d => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:43.991049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0713af-7d97-4470-9446-69d558fa2b6b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:44.032079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:44.032258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d648-80d0-4d1d-a0d6-1bae3a52ccd0 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.032529 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d648-80d0-4d1d-a0d6-1bae3a52ccd0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.032711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84d648-80d0-4d1d-a0d6-1bae3a52ccd0 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.034145 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 201/414] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:44.044948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b5011-1d80-4140-8492-51e807e9f421 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.045249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b5011-1d80-4140-8492-51e807e9f421 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.045516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b5011-1d80-4140-8492-51e807e9f421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.073544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-366b5011-1d80-4140-8492-51e807e9f421 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:44.073544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-366b5011-1d80-4140-8492-51e807e9f421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:44.157580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0713af-7d97-4470-9446-69d558fa2b6b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:44.166306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 202/415] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:44.191283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b5011-1d80-4140-8492-51e807e9f421 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:44.194022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c940b-acad-4b52-b80a-9844560a81e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:44.194998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c940b-acad-4b52-b80a-9844560a81e5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.195285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c940b-acad-4b52-b80a-9844560a81e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.195584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c940b-acad-4b52-b80a-9844560a81e5 None admin] PATH_INFO: `/v3/projects/c42afd51dd0d44aba511a26dccae48ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.196804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c940b-acad-4b52-b80a-9844560a81e5 None admin] RBAC: Authorizing `identity:get_project(project_id=c42afd51dd0d44aba511a26dccae48ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:44.198273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c940b-acad-4b52-b80a-9844560a81e5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:44.200190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 214/416] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:43 2022] GET /identity/v3/projects/c42afd51dd0d44aba511a26dccae48ce => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:44.205788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c940b-acad-4b52-b80a-9844560a81e5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:44.243702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:44.293467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:44.308897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c940b-acad-4b52-b80a-9844560a81e5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:44.343921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b43c6-09d7-490f-a3a0-4a57b6c479ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:44.344924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b43c6-09d7-490f-a3a0-4a57b6c479ed None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.345312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b43c6-09d7-490f-a3a0-4a57b6c479ed None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.345690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b43c6-09d7-490f-a3a0-4a57b6c479ed None admin] PATH_INFO: `/v3/projects/c42afd51dd0d44aba511a26dccae48ce/groups/8b3041d67d4e4fa3853ab2207d2d695d/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.350487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b5011-1d80-4140-8492-51e807e9f421 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:44.352533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b43c6-09d7-490f-a3a0-4a57b6c479ed None admin] RBAC: Authorizing `identity:create_grant(project_id=c42afd51dd0d44aba511a26dccae48ce, group_id=8b3041d67d4e4fa3853ab2207d2d695d, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:44.354002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b43c6-09d7-490f-a3a0-4a57b6c479ed None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:44.395866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67200cfe-4913-4cce-b781-3ec8c838173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:44.396878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67200cfe-4913-4cce-b781-3ec8c838173f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.397167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67200cfe-4913-4cce-b781-3ec8c838173f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.397443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67200cfe-4913-4cce-b781-3ec8c838173f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.414431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67200cfe-4913-4cce-b781-3ec8c838173f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-67200cfe-4913-4cce-b781-3ec8c838173f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:44.422842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:44.423838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:44.424631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:44.427390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 203/417] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:44.433962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67200cfe-4913-4cce-b781-3ec8c838173f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:44.457766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 215/418] 10.222.0.98 () {58 vars in 1442 bytes} [Fri Jul 15 15:32:44 2022] PUT /identity/v3/projects/c42afd51dd0d44aba511a26dccae48ce/groups/8b3041d67d4e4fa3853ab2207d2d695d/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:32:44.553936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:44.563059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67200cfe-4913-4cce-b781-3ec8c838173f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:44.609285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43686396-209e-4e60-aa8c-cd4f18061c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:44.610342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43686396-209e-4e60-aa8c-cd4f18061c3a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.610826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43686396-209e-4e60-aa8c-cd4f18061c3a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.611328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43686396-209e-4e60-aa8c-cd4f18061c3a None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.612307 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43686396-209e-4e60-aa8c-cd4f18061c3a None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:44.612808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43686396-209e-4e60-aa8c-cd4f18061c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:44.614457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43686396-209e-4e60-aa8c-cd4f18061c3a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:44.624648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 204/419] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:44.631137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b43c6-09d7-490f-a3a0-4a57b6c479ed None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:44.762208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b43c6-09d7-490f-a3a0-4a57b6c479ed None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:44.800653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3df93a-5dae-4932-a889-14ecdd4f8e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:44.801313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3df93a-5dae-4932-a889-14ecdd4f8e09 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.801602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3df93a-5dae-4932-a889-14ecdd4f8e09 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.801883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3df93a-5dae-4932-a889-14ecdd4f8e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.813517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04aaff-d31b-4508-84d8-9f4fe1bcdf67 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.814417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04aaff-d31b-4508-84d8-9f4fe1bcdf67 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.814874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04aaff-d31b-4508-84d8-9f4fe1bcdf67 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.816549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 205/420] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:44.819819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:44.822111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e82e1-155c-4e85-9714-d5bc15106d76 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:44.822407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e82e1-155c-4e85-9714-d5bc15106d76 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:44.822691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e82e1-155c-4e85-9714-d5bc15106d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:44.845853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ce0e82e1-155c-4e85-9714-d5bc15106d76 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:44.847941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ce0e82e1-155c-4e85-9714-d5bc15106d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:44.908459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3df93a-5dae-4932-a889-14ecdd4f8e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:44.910744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3df93a-5dae-4932-a889-14ecdd4f8e09 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:44.939255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 206/421] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:44.974210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e82e1-155c-4e85-9714-d5bc15106d76 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:45.042365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 216/422] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:45.047740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3df93a-5dae-4932-a889-14ecdd4f8e09 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:45.136779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:45.144393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:45.166315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e82e1-155c-4e85-9714-d5bc15106d76 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:45.215074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89533093-e379-4565-96b3-5678f3d455cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:45.215921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89533093-e379-4565-96b3-5678f3d455cb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:45.216206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89533093-e379-4565-96b3-5678f3d455cb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:45.216483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89533093-e379-4565-96b3-5678f3d455cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:45.219849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3df93a-5dae-4932-a889-14ecdd4f8e09 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:45.227350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89533093-e379-4565-96b3-5678f3d455cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-89533093-e379-4565-96b3-5678f3d455cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:45.235748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:45.236609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:45.237364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 207/423] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:45.248285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89533093-e379-4565-96b3-5678f3d455cb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:45.266959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377aa832-7e0c-4670-ac23-152a724c3d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:45.267866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377aa832-7e0c-4670-ac23-152a724c3d16 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:45.268152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377aa832-7e0c-4670-ac23-152a724c3d16 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:45.268429 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377aa832-7e0c-4670-ac23-152a724c3d16 None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:45.272660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377aa832-7e0c-4670-ac23-152a724c3d16 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:45.274151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377aa832-7e0c-4670-ac23-152a724c3d16 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:45.276600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 217/424] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:45 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:45.282005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377aa832-7e0c-4670-ac23-152a724c3d16 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:45.352264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89533093-e379-4565-96b3-5678f3d455cb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:45.402293 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377aa832-7e0c-4670-ac23-152a724c3d16 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:45.419896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:45.420088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:45.420783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:45.421068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:45.421344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:45.422102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:45.422408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:45.423824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:45.442645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a43416-024b-4a99-984a-f8b1b45caa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:45.443720 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a43416-024b-4a99-984a-f8b1b45caa79 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:45.444008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a43416-024b-4a99-984a-f8b1b45caa79 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:45.444340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a43416-024b-4a99-984a-f8b1b45caa79 None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:45.445552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a43416-024b-4a99-984a-f8b1b45caa79 None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:45.447011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a43416-024b-4a99-984a-f8b1b45caa79 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:45.449088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 218/425] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:45 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:45.449383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 208/426] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:45.458057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a43416-024b-4a99-984a-f8b1b45caa79 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:45.460202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:45.481997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:45.592580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a43416-024b-4a99-984a-f8b1b45caa79 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:45.662074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594060fb-174c-4ac2-a062-39c3e1869d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:45.664422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594060fb-174c-4ac2-a062-39c3e1869d2f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:45.666040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594060fb-174c-4ac2-a062-39c3e1869d2f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:45.667751 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594060fb-174c-4ac2-a062-39c3e1869d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:45.674950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fab74d-0609-4517-93cd-014bd34ad7a2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:45.716813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3c95-d88e-4cdf-8e38-21db3fc230b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:45.717815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3c95-d88e-4cdf-8e38-21db3fc230b0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:45.718329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3c95-d88e-4cdf-8e38-21db3fc230b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:45.718813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3c95-d88e-4cdf-8e38-21db3fc230b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:45.719941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3c95-d88e-4cdf-8e38-21db3fc230b0 None admin] RBAC: Adding query filter params (group.id=d71a2f7b114c4155badd9ac70c64b325, role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=b7fddf3939de4bb99741de8e3307565f) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:45.720459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3c95-d88e-4cdf-8e38-21db3fc230b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:45.722106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3c95-d88e-4cdf-8e38-21db3fc230b0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:45.729622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 209/427] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:45 2022] GET /identity/v3/role_assignments?group.id=d71a2f7b114c4155badd9ac70c64b325&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 611 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:45.783413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:45.805341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a06b-7a28-4b11-8c7e-36d136b7e82f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:45.805658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a06b-7a28-4b11-8c7e-36d136b7e82f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:45.805912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226a06b-7a28-4b11-8c7e-36d136b7e82f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:45.807334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 210/428] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:45.814151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d6c5b-06e6-4d41-b2f6-61469e1881f0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:45.814783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d6c5b-06e6-4d41-b2f6-61469e1881f0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:45.814783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d6c5b-06e6-4d41-b2f6-61469e1881f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:45.818140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594060fb-174c-4ac2-a062-39c3e1869d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:45.819762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594060fb-174c-4ac2-a062-39c3e1869d2f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:45.835704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-055d6c5b-06e6-4d41-b2f6-61469e1881f0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:45.836937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-055d6c5b-06e6-4d41-b2f6-61469e1881f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:45.942323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 211/429] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:45.966933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d6c5b-06e6-4d41-b2f6-61469e1881f0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:45.969261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:45.983927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 219/430] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:45.990979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594060fb-174c-4ac2-a062-39c3e1869d2f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:46.096395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:46.104741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d6c5b-06e6-4d41-b2f6-61469e1881f0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:46.124835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594060fb-174c-4ac2-a062-39c3e1869d2f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:46.158774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bb291-cc34-4aaa-8179-8ed516ee64d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:46.160117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bb291-cc34-4aaa-8179-8ed516ee64d7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.160578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bb291-cc34-4aaa-8179-8ed516ee64d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.161029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bb291-cc34-4aaa-8179-8ed516ee64d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.172077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763e549-3d2d-4e7b-9324-c715af05922c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:46.172955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763e549-3d2d-4e7b-9324-c715af05922c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.173248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763e549-3d2d-4e7b-9324-c715af05922c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.173528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763e549-3d2d-4e7b-9324-c715af05922c None admin] PATH_INFO: `/v3/projects/b7fddf3939de4bb99741de8e3307565f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.175385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763e549-3d2d-4e7b-9324-c715af05922c None admin] RBAC: Authorizing `identity:get_project(project_id=b7fddf3939de4bb99741de8e3307565f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:46.176861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763e549-3d2d-4e7b-9324-c715af05922c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:46.179321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bb291-cc34-4aaa-8179-8ed516ee64d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:46.179726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 220/431] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:45 2022] GET /identity/v3/projects/b7fddf3939de4bb99741de8e3307565f => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:46.185152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763e549-3d2d-4e7b-9324-c715af05922c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e18bb291-cc34-4aaa-8179-8ed516ee64d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:46.188325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:46.189251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:32:46.190017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:46.192286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 212/432] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:46.198679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bb291-cc34-4aaa-8179-8ed516ee64d7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:46.282057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:46.332119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763e549-3d2d-4e7b-9324-c715af05922c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:46.339818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bb291-cc34-4aaa-8179-8ed516ee64d7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:46.387012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7617eb-11e8-467b-8e8d-ba21686ced5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:46.388313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7617eb-11e8-467b-8e8d-ba21686ced5d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.388313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7617eb-11e8-467b-8e8d-ba21686ced5d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.388609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7617eb-11e8-467b-8e8d-ba21686ced5d None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.390122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7617eb-11e8-467b-8e8d-ba21686ced5d None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:46.392848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7617eb-11e8-467b-8e8d-ba21686ced5d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:46.394512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 221/433] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:46 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:46.395786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa55f-b1eb-4982-9a44-b857af2461d9 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.396075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa55f-b1eb-4982-9a44-b857af2461d9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.396359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa55f-b1eb-4982-9a44-b857af2461d9 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.397513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079d632-14ba-426f-a2b3-9237732f19b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:46.398109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 222/434] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:46.398615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079d632-14ba-426f-a2b3-9237732f19b0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.398907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079d632-14ba-426f-a2b3-9237732f19b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.399193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079d632-14ba-426f-a2b3-9237732f19b0 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.400503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079d632-14ba-426f-a2b3-9237732f19b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:46.400784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa55f-b1eb-4982-9a44-b857af2461d9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:46.400886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079d632-14ba-426f-a2b3-9237732f19b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:46.405724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079d632-14ba-426f-a2b3-9237732f19b0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:46.415970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 213/435] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:32:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:46.417137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2687799-6b94-4828-b973-ecc52ae2379c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.417430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2687799-6b94-4828-b973-ecc52ae2379c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.417785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2687799-6b94-4828-b973-ecc52ae2379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.428242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:46.439251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b2687799-6b94-4828-b973-ecc52ae2379c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:46.440517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b2687799-6b94-4828-b973-ecc52ae2379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:46.530509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 214/436] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:46.531392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2687799-6b94-4828-b973-ecc52ae2379c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:46.547778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa55f-b1eb-4982-9a44-b857af2461d9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:46.589222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:46.592122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:46.592783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.593302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.593302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.594147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] RBAC: Adding query filter params (group.id=d71a2f7b114c4155badd9ac70c64b325, role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=b7fddf3939de4bb99741de8e3307565f) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:46.594459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:46.596027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:46.602974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 223/437] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:46 2022] GET /identity/v3/role_assignments?group.id=d71a2f7b114c4155badd9ac70c64b325&scope.project.id=b7fddf3939de4bb99741de8e3307565f&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:46.603807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:46.659103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2687799-6b94-4828-b973-ecc52ae2379c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:46.704131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:46.712706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b200815-942b-4c85-bfd3-76a703411757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:46.713990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200815-942b-4c85-bfd3-76a703411757 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.714465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200815-942b-4c85-bfd3-76a703411757 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.715083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200815-942b-4c85-bfd3-76a703411757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.725476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704352d5-9b1a-415f-85fc-4c5979c6b866 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:46.767402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fa3f0-5b3f-4be3-b6ac-81082025417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:46.768470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fa3f0-5b3f-4be3-b6ac-81082025417d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.768580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fa3f0-5b3f-4be3-b6ac-81082025417d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.768862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fa3f0-5b3f-4be3-b6ac-81082025417d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.780420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fa3f0-5b3f-4be3-b6ac-81082025417d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-808fa3f0-5b3f-4be3-b6ac-81082025417d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:46.788651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:46.789543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:46.790360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 224/438] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:46.795725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fa3f0-5b3f-4be3-b6ac-81082025417d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:46.804916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b200815-942b-4c85-bfd3-76a703411757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:46.805299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b200815-942b-4c85-bfd3-76a703411757 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:46.895903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:46.946836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fa3f0-5b3f-4be3-b6ac-81082025417d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:46.964235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 215/439] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:46.969601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b200815-942b-4c85-bfd3-76a703411757 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:46.997349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:46.998104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:46.998307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:46.998588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:46.999597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:47.000017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.001471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.012984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 225/440] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:47.019539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.023462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.087657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b200815-942b-4c85-bfd3-76a703411757 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.121652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d1585-bd4f-4772-8e64-92b1ae8ebf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.122509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1585-bd4f-4772-8e64-92b1ae8ebf3b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.122797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1585-bd4f-4772-8e64-92b1ae8ebf3b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.123071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d1585-bd4f-4772-8e64-92b1ae8ebf3b None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.124291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d1585-bd4f-4772-8e64-92b1ae8ebf3b None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.125744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d1585-bd4f-4772-8e64-92b1ae8ebf3b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.127454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 216/441] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:46 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:47.128646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cdc80-8dad-4bcd-bca7-3690587b07fb None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.128936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cdc80-8dad-4bcd-bca7-3690587b07fb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.129218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cdc80-8dad-4bcd-bca7-3690587b07fb None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.130590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 217/442] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:47.132720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cdc80-8dad-4bcd-bca7-3690587b07fb None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.132841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87786017-3231-475a-8ab5-7ada2c60afb0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.156857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.168106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c0cfa-2ea9-48f8-b7ca-2bfeed0d3a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.168792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0cfa-2ea9-48f8-b7ca-2bfeed0d3a38 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.169077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0cfa-2ea9-48f8-b7ca-2bfeed0d3a38 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.169358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c0cfa-2ea9-48f8-b7ca-2bfeed0d3a38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.237822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cdc80-8dad-4bcd-bca7-3690587b07fb None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.244245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c0cfa-2ea9-48f8-b7ca-2bfeed0d3a38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.245714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c0cfa-2ea9-48f8-b7ca-2bfeed0d3a38 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.248446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.272845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8b9be-5b27-4acf-a33a-2c091fa5867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.273694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8b9be-5b27-4acf-a33a-2c091fa5867e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.273983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8b9be-5b27-4acf-a33a-2c091fa5867e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.274264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8b9be-5b27-4acf-a33a-2c091fa5867e None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.275897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8b9be-5b27-4acf-a33a-2c091fa5867e None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.277338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8b9be-5b27-4acf-a33a-2c091fa5867e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.278974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 218/443] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:47.280202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9784697-06ba-4d99-a55a-436a961b2872 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.280495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9784697-06ba-4d99-a55a-436a961b2872 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.280775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9784697-06ba-4d99-a55a-436a961b2872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.297416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c9784697-06ba-4d99-a55a-436a961b2872 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:47.298608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c9784697-06ba-4d99-a55a-436a961b2872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:47.349149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 226/444] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:47.350467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c0cfa-2ea9-48f8-b7ca-2bfeed0d3a38 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.368214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 219/445] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:47.369038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9784697-06ba-4d99-a55a-436a961b2872 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.403660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.454250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c0cfa-2ea9-48f8-b7ca-2bfeed0d3a38 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.457135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.481290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9784697-06ba-4d99-a55a-436a961b2872 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.490015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.491659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.491947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.492225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.493371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] RBAC: Adding query filter params (group.id=d71a2f7b114c4155badd9ac70c64b325, role.id=0892f8baef1c4b61b13187ba945d2ad5, scope.project.id=366bead620044a59b9ae769237db8568) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:47.493687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.495144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.500954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 227/446] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/role_assignments?group.id=d71a2f7b114c4155badd9ac70c64b325&scope.project.id=366bead620044a59b9ae769237db8568&role.id=0892f8baef1c4b61b13187ba945d2ad5 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:47.501751 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.515826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db33bf6-684d-4e44-ad2e-f65a2d5579db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.516741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33bf6-684d-4e44-ad2e-f65a2d5579db None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.517020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33bf6-684d-4e44-ad2e-f65a2d5579db None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.517301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33bf6-684d-4e44-ad2e-f65a2d5579db None admin] PATH_INFO: `/v3/projects/366bead620044a59b9ae769237db8568` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.518600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db33bf6-684d-4e44-ad2e-f65a2d5579db None admin] RBAC: Authorizing `identity:get_project(project_id=366bead620044a59b9ae769237db8568)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.520130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db33bf6-684d-4e44-ad2e-f65a2d5579db None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.522678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 220/447] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/projects/366bead620044a59b9ae769237db8568 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:47.528528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db33bf6-684d-4e44-ad2e-f65a2d5579db None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.624819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9dc0f3-6a46-4799-9b01-c3ed32b1a5bf None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.625431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.643967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db33bf6-684d-4e44-ad2e-f65a2d5579db None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.666332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8f92d-a423-49c1-9a71-dc5905e87c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.667161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f92d-a423-49c1-9a71-dc5905e87c08 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.667457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f92d-a423-49c1-9a71-dc5905e87c08 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.667738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8f92d-a423-49c1-9a71-dc5905e87c08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.680247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8f92d-a423-49c1-9a71-dc5905e87c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:47.687818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74b63-dd09-4b70-ab5c-a874b48fc64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9be8f92d-a423-49c1-9a71-dc5905e87c08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:47.687976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:47.689007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:32:47.690003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:47.692103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 228/448] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:47.692195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74b63-dd09-4b70-ab5c-a874b48fc64f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.692195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74b63-dd09-4b70-ab5c-a874b48fc64f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.692195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74b63-dd09-4b70-ab5c-a874b48fc64f None admin] PATH_INFO: `/v3/groups/d71a2f7b114c4155badd9ac70c64b325` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.692759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb74b63-dd09-4b70-ab5c-a874b48fc64f None admin] RBAC: Authorizing `identity:get_group(group_id=d71a2f7b114c4155badd9ac70c64b325)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.694806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb74b63-dd09-4b70-ab5c-a874b48fc64f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.696351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8f92d-a423-49c1-9a71-dc5905e87c08 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.696696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 221/449] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/groups/d71a2f7b114c4155badd9ac70c64b325 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:47.702454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74b63-dd09-4b70-ab5c-a874b48fc64f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.732956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.821675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74b63-dd09-4b70-ab5c-a874b48fc64f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.826905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8f92d-a423-49c1-9a71-dc5905e87c08 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:47.859858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.860417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.860709 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.860989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.861822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] RBAC: Adding query filter params (group.id=d71a2f7b114c4155badd9ac70c64b325, role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=366bead620044a59b9ae769237db8568) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:47.862137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.863648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.867587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:47.868478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.868838 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.869116 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.869860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:47.870170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:47.870517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 222/450] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/role_assignments?group.id=d71a2f7b114c4155badd9ac70c64b325&scope.project.id=366bead620044a59b9ae769237db8568&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:47.871581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:47.880538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 229/451] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:47.887689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:47.911506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.994663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049819-4ec7-4936-8318-ff90aa8c8cdd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:48.031417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e24562-73af-4818-8d8b-f962840a7282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:48.032068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e24562-73af-4818-8d8b-f962840a7282 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:48.032358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e24562-73af-4818-8d8b-f962840a7282 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:48.033214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e24562-73af-4818-8d8b-f962840a7282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:48.102434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e24562-73af-4818-8d8b-f962840a7282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:48.103950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e24562-73af-4818-8d8b-f962840a7282 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:48.124784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:48.203114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 223/452] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:48.208692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:48.301733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:48.310782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a629ee-4813-44d2-b77f-3c8965905daf None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:48.344209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bfc2a-ee29-42e6-8258-e0b3e7cf439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:48.345062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bfc2a-ee29-42e6-8258-e0b3e7cf439f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:48.345342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bfc2a-ee29-42e6-8258-e0b3e7cf439f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:48.345615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bfc2a-ee29-42e6-8258-e0b3e7cf439f None admin] PATH_INFO: `/v3/projects/c42afd51dd0d44aba511a26dccae48ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:48.346784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bfc2a-ee29-42e6-8258-e0b3e7cf439f None admin] RBAC: Authorizing `identity:get_project(project_id=c42afd51dd0d44aba511a26dccae48ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:48.348281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bfc2a-ee29-42e6-8258-e0b3e7cf439f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:48.350013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 230/453] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:32:48 2022] GET /identity/v3/projects/c42afd51dd0d44aba511a26dccae48ce => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:48.355649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e24562-73af-4818-8d8b-f962840a7282 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:48.455044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e24562-73af-4818-8d8b-f962840a7282 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:48.474647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:48.488070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121919a-25a2-4946-a6b0-df64e6f70603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:48.488912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121919a-25a2-4946-a6b0-df64e6f70603 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:48.489198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121919a-25a2-4946-a6b0-df64e6f70603 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:48.489468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121919a-25a2-4946-a6b0-df64e6f70603 None admin] PATH_INFO: `/v3/groups/8b3041d67d4e4fa3853ab2207d2d695d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:48.490648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121919a-25a2-4946-a6b0-df64e6f70603 None admin] RBAC: Authorizing `identity:get_group(group_id=8b3041d67d4e4fa3853ab2207d2d695d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:48.492107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121919a-25a2-4946-a6b0-df64e6f70603 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:48.493718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 224/454] 10.222.0.98 () {56 vars in 1180 bytes} [Fri Jul 15 15:32:48 2022] GET /identity/v3/groups/8b3041d67d4e4fa3853ab2207d2d695d => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:48.499694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bfc2a-ee29-42e6-8258-e0b3e7cf439f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:48.598156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bfc2a-ee29-42e6-8258-e0b3e7cf439f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:48.631655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9cfdf-8bbe-4999-9626-8f8876f590c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:48.632303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9cfdf-8bbe-4999-9626-8f8876f590c8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:48.632592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9cfdf-8bbe-4999-9626-8f8876f590c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:48.632876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9cfdf-8bbe-4999-9626-8f8876f590c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:48.633695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9cfdf-8bbe-4999-9626-8f8876f590c8 None admin] RBAC: Adding query filter params (group.id=8b3041d67d4e4fa3853ab2207d2d695d, role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=c42afd51dd0d44aba511a26dccae48ce) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:48.634009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9cfdf-8bbe-4999-9626-8f8876f590c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:48.635468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9cfdf-8bbe-4999-9626-8f8876f590c8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:48.641077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 231/455] 10.222.0.98 () {56 vars in 1376 bytes} [Fri Jul 15 15:32:48 2022] GET /identity/v3/role_assignments?group.id=8b3041d67d4e4fa3853ab2207d2d695d&scope.project.id=c42afd51dd0d44aba511a26dccae48ce&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:50.990002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c8ead-5222-4c3c-a90f-871a2be566b3 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:50.991069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c8ead-5222-4c3c-a90f-871a2be566b3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:50.991069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c8ead-5222-4c3c-a90f-871a2be566b3 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:50.992743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 225/456] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:50.998342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208352-6d05-490d-9355-80bdb47838bd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:50.999176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208352-6d05-490d-9355-80bdb47838bd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:50.999684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208352-6d05-490d-9355-80bdb47838bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.017453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c0208352-6d05-490d-9355-80bdb47838bd None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:51.018973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c0208352-6d05-490d-9355-80bdb47838bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:51.041779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:51.125937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 232/457] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:51.162756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c8ead-5222-4c3c-a90f-871a2be566b3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:51.333033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c8ead-5222-4c3c-a90f-871a2be566b3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:51.383874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614264-4c6b-48a3-949e-d87d1c0bc19c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.384513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614264-4c6b-48a3-949e-d87d1c0bc19c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.385016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43614264-4c6b-48a3-949e-d87d1c0bc19c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.386848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 233/458] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:51.394506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:51.394506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.394506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.394506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.400910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc31f-4f49-43ec-8c52-180086a2f89b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.401199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc31f-4f49-43ec-8c52-180086a2f89b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.401474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc31f-4f49-43ec-8c52-180086a2f89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.433457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:51.433651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:51.460933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f57fc31f-4f49-43ec-8c52-180086a2f89b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:51.462105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f57fc31f-4f49-43ec-8c52-180086a2f89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:51.497227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:51.498128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:51.498931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:51.498931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:51.498931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:51.498931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:51.501912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 226/459] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:51.508582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:51.560121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:51.623067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493266b3-e967-43dc-bee4-e2241c6bb5cd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:51.646625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 234/460] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:51.648397 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a0bc-f857-4810-b125-6dbc011ce54e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.648889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a0bc-f857-4810-b125-6dbc011ce54e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.649309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038a0bc-f857-4810-b125-6dbc011ce54e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.651367 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 235/461] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:51.657034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c01bed-962a-492d-ae60-bd021f4d6345 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.657491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c01bed-962a-492d-ae60-bd021f4d6345 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.658165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c01bed-962a-492d-ae60-bd021f4d6345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.663985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:51.665037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.665416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.666709 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.667596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:51.667991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:51.669499 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:51.679280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-55c01bed-962a-492d-ae60-bd021f4d6345 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:51.681756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-55c01bed-962a-492d-ae60-bd021f4d6345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:51.683927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 227/462] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:51.685273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:51.771283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 236/463] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:51.773257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c01bed-962a-492d-ae60-bd021f4d6345 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:51.791743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:51.824584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:51.827988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d7edc-3574-4b64-9fea-041e21ba878f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:51.864141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:51.864936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.865228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.865540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.876535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:51.877940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:51.894095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c01bed-962a-492d-ae60-bd021f4d6345 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:51.896172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:51.897514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:51.898521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:51.898521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:51.898521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:51.898521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:51.898521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 228/464] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:51.899267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:51.937100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28c07-a743-495b-84ef-02f5d319d1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:51.937892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28c07-a743-495b-84ef-02f5d319d1e6 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.938339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28c07-a743-495b-84ef-02f5d319d1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.938693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28c07-a743-495b-84ef-02f5d319d1e6 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.939508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce28c07-a743-495b-84ef-02f5d319d1e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:51.941039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce28c07-a743-495b-84ef-02f5d319d1e6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:52.007805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8267a-75cb-444c-930a-c12754fb7cd2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:52.041700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc26fb-943e-40a6-8bb8-a7eeeb975c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:52.042489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc26fb-943e-40a6-8bb8-a7eeeb975c9b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.042777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc26fb-943e-40a6-8bb8-a7eeeb975c9b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.043056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc26fb-943e-40a6-8bb8-a7eeeb975c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.055771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc26fb-943e-40a6-8bb8-a7eeeb975c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:52.063471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc26fb-943e-40a6-8bb8-a7eeeb975c9b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:52.063666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 237/465] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:51 2022] POST /identity/v3/users => generated 246 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:52.063666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28c07-a743-495b-84ef-02f5d319d1e6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-34dc26fb-943e-40a6-8bb8-a7eeeb975c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:52.088804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:52.089892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:52.091006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:52.091006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:52.091006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:52.091006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:52.092900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 229/466] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:52.094661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e79e94-9ec4-4f43-a960-f0c3016621e9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.095116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e79e94-9ec4-4f43-a960-f0c3016621e9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.095564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e79e94-9ec4-4f43-a960-f0c3016621e9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.100756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 230/467] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:52.100756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e79e94-9ec4-4f43-a960-f0c3016621e9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:52.103941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:52.157856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:52.195357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce28c07-a743-495b-84ef-02f5d319d1e6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:52.234465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdb43d-4983-4d76-bdd6-0bfe95f0fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:52.235175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdb43d-4983-4d76-bdd6-0bfe95f0fb39 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.235496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdb43d-4983-4d76-bdd6-0bfe95f0fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.235770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdb43d-4983-4d76-bdd6-0bfe95f0fb39 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.236499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcdb43d-4983-4d76-bdd6-0bfe95f0fb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:52.236807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcdb43d-4983-4d76-bdd6-0bfe95f0fb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:52.238188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcdb43d-4983-4d76-bdd6-0bfe95f0fb39 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:52.245309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e79e94-9ec4-4f43-a960-f0c3016621e9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:52.252012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 238/468] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:52.257727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee21355-d082-4e7e-877e-3ab96cb4a778 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.257727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee21355-d082-4e7e-877e-3ab96cb4a778 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.257727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee21355-d082-4e7e-877e-3ab96cb4a778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.273126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5ee21355-d082-4e7e-877e-3ab96cb4a778 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:52.274324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5ee21355-d082-4e7e-877e-3ab96cb4a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:52.292366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:52.293036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.293319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.293624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.294346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:52.294649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:52.296077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:52.311371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 231/469] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:32:52.311371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:52.362957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 239/470] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:52.363840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee21355-d082-4e7e-877e-3ab96cb4a778 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:52.406867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:52.442595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:52.449018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3aa6-b6c6-4847-b128-46efe46bdeca None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:52.486984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee21355-d082-4e7e-877e-3ab96cb4a778 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:52.492429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cab73-b465-4c81-b70b-0657ae7b66b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:52.493179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cab73-b465-4c81-b70b-0657ae7b66b1 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.493466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cab73-b465-4c81-b70b-0657ae7b66b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.493744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cab73-b465-4c81-b70b-0657ae7b66b1 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.494444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cab73-b465-4c81-b70b-0657ae7b66b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:52.495913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cab73-b465-4c81-b70b-0657ae7b66b1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:52.526048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d6575-5b1c-487a-9b40-6500b0fee2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:52.537833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d6575-5b1c-487a-9b40-6500b0fee2b3 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.537833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d6575-5b1c-487a-9b40-6500b0fee2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.537833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d6575-5b1c-487a-9b40-6500b0fee2b3 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.537833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d6575-5b1c-487a-9b40-6500b0fee2b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:52.537833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d6575-5b1c-487a-9b40-6500b0fee2b3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:52.590211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 232/471] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:52 2022] POST /identity/v3/users => generated 248 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:52.591476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cab73-b465-4c81-b70b-0657ae7b66b1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:52.630209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 240/472] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:52 2022] POST /identity/v3/users => generated 248 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:52.679708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39a904-9a99-4088-82ea-9f0cdd0f09d6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.680006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39a904-9a99-4088-82ea-9f0cdd0f09d6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.680244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39a904-9a99-4088-82ea-9f0cdd0f09d6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.682060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 241/473] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:32:52.692853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263696fc-0240-44fc-9922-b0a42d74b591 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.693162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263696fc-0240-44fc-9922-b0a42d74b591 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.693468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263696fc-0240-44fc-9922-b0a42d74b591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.731366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cab73-b465-4c81-b70b-0657ae7b66b1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:52.744498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-263696fc-0240-44fc-9922-b0a42d74b591 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:52.745755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-263696fc-0240-44fc-9922-b0a42d74b591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:52.763762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:52.774585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:52.776092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:52.783963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:52.783963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:52.789228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:52.790885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:52.819223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:52.820242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:52.821070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:52.821070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:52.821070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:52.821070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:52.821070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 233/474] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:52.827156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:52.873351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 242/475] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:52.898231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263696fc-0240-44fc-9922-b0a42d74b591 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:52.961801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b840c0-90ae-4f60-89df-8fe7447d0985 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:52.976818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:53.010101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:53.010783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:53.011616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:53.011905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:53.013030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:53.013347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:53.015816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:53.032849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 234/476] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:53.039774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:53.049302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263696fc-0240-44fc-9922-b0a42d74b591 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:53.080758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:53.087177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:53.087902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:53.088262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:53.088541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:53.089287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:53.090719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:53.137235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.sql.core [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 15:32:53.137235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 15:32:53.137235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [parameters: {'id': '0194fea36a1b434fabf16b673b3872c4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 15:32:53.137235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87026) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application result.read() Jul 15 15:32:53.161904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return ref.to_dict() Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:32:53.163373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application yield resource Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise self.value Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application session.commit() Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 15:32:53.164286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.session.flush() Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application compat.raise_( Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 15:32:53.165656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:32:53.167475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application result.read() Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:32:53.169552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [parameters: {'id': '0194fea36a1b434fabf16b673b3872c4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:53.171555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 15:32:53.173550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:53.180424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 243/477] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Jul 15 15:32:52 2022] POST /identity/v3/roles => generated 144 bytes in 273 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:53.180424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:53.206859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df999f-783a-42c1-8ba9-9b6e02d95e69 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:53.287008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995df281-cbe8-418b-89d4-fa31e984ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:53.287663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995df281-cbe8-418b-89d4-fa31e984ccaa None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:53.289397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995df281-cbe8-418b-89d4-fa31e984ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:53.289679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995df281-cbe8-418b-89d4-fa31e984ccaa None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:53.292047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995df281-cbe8-418b-89d4-fa31e984ccaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:53.293518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995df281-cbe8-418b-89d4-fa31e984ccaa None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:53.345470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9353b3-26fb-4992-9cc4-404f8d58b662 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:53.355370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:53.366865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 235/478] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:53 2022] POST /identity/v3/users => generated 249 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:32:53.402055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd652f61-f37a-4a08-ac4a-b397c2dfda78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:53.403421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd652f61-f37a-4a08-ac4a-b397c2dfda78 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:53.403987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd652f61-f37a-4a08-ac4a-b397c2dfda78 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:53.404510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd652f61-f37a-4a08-ac4a-b397c2dfda78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:53.418796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd652f61-f37a-4a08-ac4a-b397c2dfda78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-bd652f61-f37a-4a08-ac4a-b397c2dfda78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:53.428992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:53.429933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:32:53.430696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:53.434222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 244/479] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:32:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:53.442173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995df281-cbe8-418b-89d4-fa31e984ccaa None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:53.572289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:53.586839 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995df281-cbe8-418b-89d4-fa31e984ccaa None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:53.623374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf6897-a00f-46a1-9cd5-1deffb7ad558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:53.624043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6897-a00f-46a1-9cd5-1deffb7ad558 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:53.624331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6897-a00f-46a1-9cd5-1deffb7ad558 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:53.624607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6897-a00f-46a1-9cd5-1deffb7ad558 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:53.625345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf6897-a00f-46a1-9cd5-1deffb7ad558 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:53.625654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf6897-a00f-46a1-9cd5-1deffb7ad558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:53.627045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf6897-a00f-46a1-9cd5-1deffb7ad558 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:53.636406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 236/480] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:54.606952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f7f88-b940-4be6-bd02-47cd6397d03b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:54.607746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f7f88-b940-4be6-bd02-47cd6397d03b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:54.607746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f7f88-b940-4be6-bd02-47cd6397d03b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:54.609236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 245/481] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:54.614059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b923ebb-73d6-4bac-8d1f-99540480aead None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:54.614675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b923ebb-73d6-4bac-8d1f-99540480aead None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:54.614675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b923ebb-73d6-4bac-8d1f-99540480aead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:54.636271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2b923ebb-73d6-4bac-8d1f-99540480aead None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:54.637477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2b923ebb-73d6-4bac-8d1f-99540480aead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:54.718568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 237/482] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:54.754153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f7f88-b940-4be6-bd02-47cd6397d03b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:54.892969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f7f88-b940-4be6-bd02-47cd6397d03b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:54.937738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81adc24-f05c-4f65-95ee-07215dd2319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:54.938538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81adc24-f05c-4f65-95ee-07215dd2319f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:54.938826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81adc24-f05c-4f65-95ee-07215dd2319f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:54.939104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81adc24-f05c-4f65-95ee-07215dd2319f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:54.959024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81adc24-f05c-4f65-95ee-07215dd2319f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-f81adc24-f05c-4f65-95ee-07215dd2319f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:54.967636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:54.971576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:54.972366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 246/483] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:32:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:32:54.974771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b923ebb-73d6-4bac-8d1f-99540480aead None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:55.079415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:55.107507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b923ebb-73d6-4bac-8d1f-99540480aead None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:55.156971 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:55.157648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.157934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.158214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.158956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:55.159262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:55.160661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:55.175016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 238/484] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:32:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:55.180519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81adc24-f05c-4f65-95ee-07215dd2319f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:55.228482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:55.329114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81adc24-f05c-4f65-95ee-07215dd2319f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:55.380571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba38763-3e4c-44e9-bf48-5a55acf8633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:55.381754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba38763-3e4c-44e9-bf48-5a55acf8633f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.382242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba38763-3e4c-44e9-bf48-5a55acf8633f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.382736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba38763-3e4c-44e9-bf48-5a55acf8633f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.398395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba38763-3e4c-44e9-bf48-5a55acf8633f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:55.400471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba38763-3e4c-44e9-bf48-5a55acf8633f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6ba38763-3e4c-44e9-bf48-5a55acf8633f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:55.423823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:55.424683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:55.425502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:55.425502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:55.425502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:55.425502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:55.427717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 247/485] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:55.435420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:55.468486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a68a-f2b8-40d7-a4ca-db54014fb516 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.468815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a68a-f2b8-40d7-a4ca-db54014fb516 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.469088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a68a-f2b8-40d7-a4ca-db54014fb516 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.470690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 248/486] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:55.475641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033fc0f-256c-414a-8fd1-9e3c9582b051 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.475934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033fc0f-256c-414a-8fd1-9e3c9582b051 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.476250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033fc0f-256c-414a-8fd1-9e3c9582b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.508304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0033fc0f-256c-414a-8fd1-9e3c9582b051 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:55.509444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0033fc0f-256c-414a-8fd1-9e3c9582b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:55.583797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d81260-7fb8-4831-ac6a-5c9e5cb681e7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:55.622883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e3492-a0d4-4ab3-ab3c-a0247c6b6596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:55.623553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e3492-a0d4-4ab3-ab3c-a0247c6b6596 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.624001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e3492-a0d4-4ab3-ab3c-a0247c6b6596 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.624288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e3492-a0d4-4ab3-ab3c-a0247c6b6596 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.625393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e3492-a0d4-4ab3-ab3c-a0247c6b6596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:55.625703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e3492-a0d4-4ab3-ab3c-a0247c6b6596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:55.627530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e3492-a0d4-4ab3-ab3c-a0247c6b6596 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:55.643637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 239/487] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:55.644997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4ab14-dd3c-4a02-a569-adea3a04298a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.645289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4ab14-dd3c-4a02-a569-adea3a04298a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.645565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4ab14-dd3c-4a02-a569-adea3a04298a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.647064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 240/488] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:55.648560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4ab14-dd3c-4a02-a569-adea3a04298a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:55.652027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:55.654135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 249/489] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:32:55.655528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf527b5-5b2f-422a-ad6d-90658cc3eb04 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.655824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf527b5-5b2f-422a-ad6d-90658cc3eb04 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.656107 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf527b5-5b2f-422a-ad6d-90658cc3eb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.684335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4cf527b5-5b2f-422a-ad6d-90658cc3eb04 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:55.685813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4cf527b5-5b2f-422a-ad6d-90658cc3eb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:55.689273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:55.757682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 250/490] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:55.758825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf527b5-5b2f-422a-ad6d-90658cc3eb04 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:55.769286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4ab14-dd3c-4a02-a569-adea3a04298a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:55.807716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:55.808664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.808959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.809242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.817384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:55.819537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:32:55.836808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:55.838965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:55.841190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 241/491] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:32:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:55.847332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:55.863692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf527b5-5b2f-422a-ad6d-90658cc3eb04 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:55.900880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0b3c7-dd1f-49c0-81fe-3577169b5a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:55.901685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0b3c7-dd1f-49c0-81fe-3577169b5a8b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:55.901966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0b3c7-dd1f-49c0-81fe-3577169b5a8b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:55.902247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0b3c7-dd1f-49c0-81fe-3577169b5a8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:55.910795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:55.913379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:55.915170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0b3c7-dd1f-49c0-81fe-3577169b5a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6ee0b3c7-dd1f-49c0-81fe-3577169b5a8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:55.922866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:55.924024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:32:55.924997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:55.929641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 251/492] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:32:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:55.930347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0b3c7-dd1f-49c0-81fe-3577169b5a8b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:55.959502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75070f-4930-4387-9b8a-c0fddec079c6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.000328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2ed05-9cf5-4a02-b35b-3de545c329bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.001481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2ed05-9cf5-4a02-b35b-3de545c329bc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.001980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2ed05-9cf5-4a02-b35b-3de545c329bc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.002458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2ed05-9cf5-4a02-b35b-3de545c329bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.019669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2ed05-9cf5-4a02-b35b-3de545c329bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-84f2ed05-9cf5-4a02-b35b-3de545c329bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:56.031339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:56.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:32:56.033432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:56.036683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 242/493] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:32:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:56.038633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2ed05-9cf5-4a02-b35b-3de545c329bc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.041119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0b3c7-dd1f-49c0-81fe-3577169b5a8b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.077285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.077899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.078295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.078660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.079614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:56.080018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.081573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.123334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 252/494] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:32:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:56.123334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.182782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2ed05-9cf5-4a02-b35b-3de545c329bc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.184343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:56.214113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:56.231547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.232299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08aca7e-49d6-4778-9b87-1bb25dc41d47 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.232616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.233047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.233471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.234733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:56.236141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.237705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.249235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 243/495] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:56.250799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.276202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.277050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.277623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.277905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.278649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:56.278956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.280809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.295475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 253/496] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:56.295475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.358528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d1639-f525-4668-9f32-98de32e2e8ba None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.393668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db18434-e094-40c3-bf30-c5332a8415ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.394313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db18434-e094-40c3-bf30-c5332a8415ab None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.394592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db18434-e094-40c3-bf30-c5332a8415ab None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.394865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db18434-e094-40c3-bf30-c5332a8415ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.405963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496991-da3f-4173-8c73-093cc3a30665 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.426650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:56.441213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd66a66-d1db-4c3e-b750-c0f604c041e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.442014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd66a66-d1db-4c3e-b750-c0f604c041e1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.442303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd66a66-d1db-4c3e-b750-c0f604c041e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.451998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd66a66-d1db-4c3e-b750-c0f604c041e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.452133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:56.453730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd66a66-d1db-4c3e-b750-c0f604c041e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.455206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd66a66-d1db-4c3e-b750-c0f604c041e1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.462709 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db18434-e094-40c3-bf30-c5332a8415ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.464179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db18434-e094-40c3-bf30-c5332a8415ab None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-afd66a66-d1db-4c3e-b750-c0f604c041e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:56.472376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:56.478219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 254/497] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f995442-5148-4418-9e1b-e4494f9dec11 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f995442-5148-4418-9e1b-e4494f9dec11 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f995442-5148-4418-9e1b-e4494f9dec11 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.479034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 255/498] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:56.479528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f995442-5148-4418-9e1b-e4494f9dec11 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.569917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 244/499] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:56.571500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356be2-ce6f-409c-ad92-1cd1cf7ac2b6 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.571788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356be2-ce6f-409c-ad92-1cd1cf7ac2b6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.572062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356be2-ce6f-409c-ad92-1cd1cf7ac2b6 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.573324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 245/500] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:56.574003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356be2-ce6f-409c-ad92-1cd1cf7ac2b6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.587956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f995442-5148-4418-9e1b-e4494f9dec11 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.625790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8183f-ddcc-414f-bdbe-d021cbd72cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.626614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8183f-ddcc-414f-bdbe-d021cbd72cd3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.626898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8183f-ddcc-414f-bdbe-d021cbd72cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.627177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8183f-ddcc-414f-bdbe-d021cbd72cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.637742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8183f-ddcc-414f-bdbe-d021cbd72cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.639228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8183f-ddcc-414f-bdbe-d021cbd72cd3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ebf8183f-ddcc-414f-bdbe-d021cbd72cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:56.656115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:56.657482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:56.658305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:56.658305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:56.658305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:56.658305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:56.658305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 256/501] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:56.658566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f564c-b175-418c-9fa3-8977c2e18b8e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.658916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f564c-b175-418c-9fa3-8977c2e18b8e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.659002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f564c-b175-418c-9fa3-8977c2e18b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.662985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:56.675598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-269f564c-b175-418c-9fa3-8977c2e18b8e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:56.676784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-269f564c-b175-418c-9fa3-8977c2e18b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:56.679763 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356be2-ce6f-409c-ad92-1cd1cf7ac2b6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.694886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:56.713895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.714545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.714827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.715098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.715833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:56.716133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.717492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.728727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 246/502] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:56.729506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.744927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 257/503] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:56.746293 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc84c0-3011-458f-9825-b3126c76cc8e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.746593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc84c0-3011-458f-9825-b3126c76cc8e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.746874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc84c0-3011-458f-9825-b3126c76cc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.766254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d8bc84c0-3011-458f-9825-b3126c76cc8e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:56.767504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d8bc84c0-3011-458f-9825-b3126c76cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:56.833594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8514848-a3e4-42ca-a948-00555d2b31dd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.835705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 258/504] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:32:56.836623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc84c0-3011-458f-9825-b3126c76cc8e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.872279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.873071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.873358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.873636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.884493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.885945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.888448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:56.903932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:56.904816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:56.905647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:56.905647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:56.905647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:56.905647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:56.905647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 247/505] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:56.905910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:56.921467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:56.937603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc84c0-3011-458f-9825-b3126c76cc8e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:56.972233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:56.972904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:56.973192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:56.973475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:56.974197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:56.974509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:56.976002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:56.987364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 259/506] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:56.988217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.017262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98f4fe-9015-49ee-b7ef-2e0e570fd0ad None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.050995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.051908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.052201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.052483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] PATH_INFO: `/v3/users/glance` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.060655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.062129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:32:57.077191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:57.078084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:57.078943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 248/507] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:57.079633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.093135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60528483-938b-4ba3-8143-f56307c34c23 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.136470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.137271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.137559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.137833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.145009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:57.149063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.150785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.153514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:57.168763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:57.170417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:57.172125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:57.172125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:57.172125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:57.172125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:57.172125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 260/508] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:57.172125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.192868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224c7c0-544d-4acb-9c9f-7f40bd11bf0b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.227502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897dc6d-0e94-4512-ac04-357919d596c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.228294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897dc6d-0e94-4512-ac04-357919d596c6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.228590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897dc6d-0e94-4512-ac04-357919d596c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.228865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897dc6d-0e94-4512-ac04-357919d596c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.238796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897dc6d-0e94-4512-ac04-357919d596c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f897dc6d-0e94-4512-ac04-357919d596c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:32:57.246491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:57.247415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:57.248182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 249/509] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:57.248841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897dc6d-0e94-4512-ac04-357919d596c6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.274855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cec197-7067-416b-8700-1619f60f814f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.308313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.308977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.309266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.309613 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.310343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:57.310654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.312083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.324254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 261/510] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:57.325061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.353299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897dc6d-0e94-4512-ac04-357919d596c6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.368927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:57.382397 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:57.388726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.388726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.388726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.389228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.397780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.399236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:32:57.414138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:57.415187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:57.416081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 250/511] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:57.416634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.432115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f654f7a-db8d-4ecf-a67f-3738066aa8d2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.466986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.467793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.468514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.471079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.471079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:57.471079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.472303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.490459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 262/512] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:57.491873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.516832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fc549-a62a-4f6b-be0a-af7a46baf2a0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.549736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.550389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.550675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.550951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.551698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:57.552005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.553956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.565385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 251/513] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:57.566159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.592865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b1ecc-a0d8-4fca-a106-95412349ce49 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.600754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:57.611760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:57.626686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.627399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.627692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.627975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.628721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:57.629030 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.630484 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.638896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 263/514] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:57.639709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.668813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9ade2-6b2c-41fc-8692-896ab94727fe None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.702119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.702897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.703177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.703485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.710542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.712019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:32:57.726372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:57.727277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:57.728242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 252/515] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:57.728693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.747050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c515d-14a6-45a1-a938-6713ccdc5ac4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.780306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.780971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.781256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.781536 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.782357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:57.782669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.784148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.802692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 264/516] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:57.803631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:57.832513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:57.836613 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:57.839151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc7a26-fad1-4119-96e9-13af87b903a1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.872595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a18c71-7eda-420b-99e8-97c6eaa452b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.873177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18c71-7eda-420b-99e8-97c6eaa452b2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.873463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18c71-7eda-420b-99e8-97c6eaa452b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.873743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18c71-7eda-420b-99e8-97c6eaa452b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.900876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c265d13-71aa-4acd-9507-622525217c63 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:57.934128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f196b55-ac37-42a0-a734-6914c4bb270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:57.934866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f196b55-ac37-42a0-a734-6914c4bb270d None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:57.935152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f196b55-ac37-42a0-a734-6914c4bb270d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:57.935456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f196b55-ac37-42a0-a734-6914c4bb270d None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:57.936154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f196b55-ac37-42a0-a734-6914c4bb270d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.937585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f196b55-ac37-42a0-a734-6914c4bb270d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:57.941242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a18c71-7eda-420b-99e8-97c6eaa452b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:57.942670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a18c71-7eda-420b-99e8-97c6eaa452b2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.033647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:58.040661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 253/517] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:58.047547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a18c71-7eda-420b-99e8-97c6eaa452b2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.047658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 265/518] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:32:57 2022] POST /identity/v3/users => generated 247 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:32:58.047658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f196b55-ac37-42a0-a734-6914c4bb270d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.155392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:58.161799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a18c71-7eda-420b-99e8-97c6eaa452b2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.164756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f196b55-ac37-42a0-a734-6914c4bb270d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.199170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.200007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.200293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.200577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.203015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.203772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.204057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.205440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.205440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:58.205440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.207765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.212592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.214573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.231781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 266/519] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:58.232331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:58.232983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:58.233786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:58.234997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:58.234997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:58.234997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:58.234997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:58.234997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 254/520] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:58.235547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.295108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:58.344478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42d3da-1a28-4858-bf7f-c0bbbfabc555 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.345704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaf027-8eca-4858-8a9f-a77c96a574c8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.382164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.382164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.382456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.382524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41857c6-f0b9-47c0-821f-678899d95d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.382780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.385044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41857c6-f0b9-47c0-821f-678899d95d9f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.385044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41857c6-f0b9-47c0-821f-678899d95d9f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.385044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41857c6-f0b9-47c0-821f-678899d95d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.394298 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.395819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.409486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:58.418615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:58.420309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:58.421139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:58.421139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:58.421139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:58.421139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:58.424495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 255/521] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:58.425142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.466338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41857c6-f0b9-47c0-821f-678899d95d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.468283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41857c6-f0b9-47c0-821f-678899d95d9f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.528023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:58.531455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188dad4-f6f5-4792-b10a-d2cd8f33d999 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.568483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.569653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.569934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.570208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.571027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=bf670978cc2e4c529093a191bc26f885) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:58.571357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.572801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.577790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 267/522] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:58.578620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 256/523] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/role_assignments?user.id=bf670978cc2e4c529093a191bc26f885&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:58.578839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41857c6-f0b9-47c0-821f-678899d95d9f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.579434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.655642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:58.710237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41857c6-f0b9-47c0-821f-678899d95d9f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.732075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385461a-bbed-4800-9431-2ef597be459f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.749354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.750172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.750576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.750942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.751798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:58.752198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.753675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.766662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 268/524] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:58.767402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.779139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.779843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.780133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.780417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.781142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:58.781452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.782905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.795505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 257/525] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:58.796645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.824946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:58.887459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f778-dd93-4cb4-b244-6cecb534c6f0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.899797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:58.905341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb87a3d-12fb-40e3-8f52-e6fa76ba06d7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:58.926343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.927575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.928108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.928687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.941142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:58.941948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.942057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:58.942243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:58.942509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:58.945391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.951685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:58.953663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:32:58.965830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:58.966700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:58.967621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:32:58.967621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:32:58.967621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:32:58.967621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:32:58.968496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:58.969313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:32:58.970161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:58.976509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 258/527] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:58.976509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:58.976842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 269/527] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:58.978052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:59.064689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:59.085056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060e00d-e9bb-4083-8080-170e158beca0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.090630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d13c40-1195-4ede-ac86-712fe8e05886 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.121647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:59.122565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:59.122854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:59.123126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:59.130275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:59.132280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:59.139791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:59.140749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:59.141283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:59.141767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:59.142819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:59.143352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:59.145014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:32:59.147698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:59.148534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 259/528] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:59.154627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:59.178761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 270/529] 10.222.0.98 () {56 vars in 1139 bytes} [Fri Jul 15 15:32:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:59.179698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:59.212814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:59.280884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d65ea-711a-41b7-8f6e-baa7cc3da0d2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.321193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:59.322209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a710d-286b-48df-ba56-605a3fb55b21 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.322474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:59.322956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:59.323473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:59.324392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:59.324915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:59.326489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:59.339469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 260/530] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:59.340609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:59.345111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:59.366512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:59.367228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:59.367563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:59.368137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:59.369246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:59.369559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:59.371507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:59.387679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 271/531] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:59.390053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:59.460894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec011cda-965e-481d-b379-941a08abcc55 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.496466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424597a-d950-4ec6-bde4-ff7869ef135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:59.497415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424597a-d950-4ec6-bde4-ff7869ef135c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:59.497915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424597a-d950-4ec6-bde4-ff7869ef135c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:59.498389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424597a-d950-4ec6-bde4-ff7869ef135c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:59.503822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:59.529785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50083bf-6f1d-4a60-952b-5f971a12bfa6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.567523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:59.569161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:59.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:59.570141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:59.577233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:59.578919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:59.582077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424597a-d950-4ec6-bde4-ff7869ef135c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:59.583689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424597a-d950-4ec6-bde4-ff7869ef135c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:59.587243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:32:59.595750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:32:59.601222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 272/532] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:32:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:32:59.602060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:59.703446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4b613-44d5-4cec-a31e-69e10ae60f77 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.718106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 261/533] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:32:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:32:59.719173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424597a-d950-4ec6-bde4-ff7869ef135c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:59.750512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:59.751953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:59.752448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:59.752978 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:59.754089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=52f18d0f5f634d84a99fc3e46963a8b4) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:59.754684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:59.756485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:59.763335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 273/534] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:32:59 2022] GET /identity/v3/role_assignments?user.id=52f18d0f5f634d84a99fc3e46963a8b4&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:32:59.764760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:59.777517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:59.913108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424597a-d950-4ec6-bde4-ff7869ef135c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.919477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:59.968271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:59.968946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:59.969232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:59.969510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:59.970301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:32:59.970614 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:59.972097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:59.981110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389727e-bc30-414e-b98d-9e775ac2d569 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:59.993192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 262/535] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:00.004539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:00.026118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:00.026946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.027227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.027539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.038600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:00.040117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:00.057677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:00.058563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:00.059413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:00.059413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:00.059413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:00.059413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:00.059413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 274/536] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:00.064455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:00.133543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:00.180501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8dc3-e484-4fbd-a20b-f84ce34a801e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:00.181671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59851508-299a-437b-a98f-8bd8c42b2eaf None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:00.216285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012ac8a-132d-4a70-8fe0-e7cec007a179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:00.216947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ac8a-132d-4a70-8fe0-e7cec007a179 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.217234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ac8a-132d-4a70-8fe0-e7cec007a179 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.217513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012ac8a-132d-4a70-8fe0-e7cec007a179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.218344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012ac8a-132d-4a70-8fe0-e7cec007a179 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=68015ac9febe42ef917ca58fae7b1c26) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:00.218658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012ac8a-132d-4a70-8fe0-e7cec007a179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:00.220034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:00.220156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012ac8a-132d-4a70-8fe0-e7cec007a179 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:00.220728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.221014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.221292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.222127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:00.222436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:00.224055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:00.226208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 263/537] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:00 2022] GET /identity/v3/role_assignments?user.id=68015ac9febe42ef917ca58fae7b1c26&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:00.231631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d07c0-310d-4a10-8811-4c9285cd5155 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.231965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d07c0-310d-4a10-8811-4c9285cd5155 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.232251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d07c0-310d-4a10-8811-4c9285cd5155 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.233833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 264/538] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:00.236993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 275/539] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:00.240151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dab031-9cd4-4ba5-8378-32dfeb170666 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.240966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dab031-9cd4-4ba5-8378-32dfeb170666 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.241251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dab031-9cd4-4ba5-8378-32dfeb170666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.242726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:00.280504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-22dab031-9cd4-4ba5-8378-32dfeb170666 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:00.281748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-22dab031-9cd4-4ba5-8378-32dfeb170666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:00.323473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:00.356627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94593f5d-b30b-4a25-a675-12c8bd66d99e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:00.381955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:00.396535 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:00.397856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.398450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.398926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.413060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:00.414563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:00.417797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 265/540] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:00.440039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:00.440916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:00.441746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 276/541] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:00.447912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dab031-9cd4-4ba5-8378-32dfeb170666 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:00.457120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:00.589196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dab031-9cd4-4ba5-8378-32dfeb170666 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:00.654491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ce0b0-749f-4ad6-a630-949af85ac0bc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:00.672127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:00.672807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.673090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.673363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.674353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:00.674467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:00.675952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:00.696871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef47e1-f3a7-4a87-afad-f8a36b9302d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:00.698039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef47e1-f3a7-4a87-afad-f8a36b9302d0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.698727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef47e1-f3a7-4a87-afad-f8a36b9302d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.699405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef47e1-f3a7-4a87-afad-f8a36b9302d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.703957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 266/542] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:00.708670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:00.724037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef47e1-f3a7-4a87-afad-f8a36b9302d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-09ef47e1-f3a7-4a87-afad-f8a36b9302d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:00.735728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:00.736643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:00.737465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:00.744936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 277/543] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:00.752308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef47e1-f3a7-4a87-afad-f8a36b9302d0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:00.753025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:00.819625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:00.869699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2201432-a23f-42ed-8ea8-74247d84a8cd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:00.908129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd200b2-9710-4303-b5bb-d3f8a7bb7c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:00.908778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef47e1-f3a7-4a87-afad-f8a36b9302d0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:00.909510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd200b2-9710-4303-b5bb-d3f8a7bb7c67 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.909998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd200b2-9710-4303-b5bb-d3f8a7bb7c67 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.910542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd200b2-9710-4303-b5bb-d3f8a7bb7c67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.911717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd200b2-9710-4303-b5bb-d3f8a7bb7c67 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=9d20de9c921942d1afeb63e87272f782) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:00.912287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd200b2-9710-4303-b5bb-d3f8a7bb7c67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:00.913974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd200b2-9710-4303-b5bb-d3f8a7bb7c67 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:00.921690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 267/544] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:00 2022] GET /identity/v3/role_assignments?user.id=9d20de9c921942d1afeb63e87272f782&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:00.923287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3896f-cbef-4198-b185-f11624901dd1 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.923902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3896f-cbef-4198-b185-f11624901dd1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.924397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3896f-cbef-4198-b185-f11624901dd1 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.926421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 268/545] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:00.942258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274256-dbb1-4bb7-a496-e6b8e60b3388 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.942593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274256-dbb1-4bb7-a496-e6b8e60b3388 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.942828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274256-dbb1-4bb7-a496-e6b8e60b3388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.953345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:00.954259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:00.954552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:00.954835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:00.955653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:00.955969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:00.957533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:00.968654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 278/546] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:00.974224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:01.046233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a7274256-dbb1-4bb7-a496-e6b8e60b3388 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:01.047506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a7274256-dbb1-4bb7-a496-e6b8e60b3388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:01.177436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb63a2-007f-41eb-880e-c3ad20bd27f0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:01.203524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:01.209378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 269/547] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:33:01.234782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:01.236510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:01.236510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:01.236510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:01.247808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:01.249355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:01.254991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7274256-dbb1-4bb7-a496-e6b8e60b3388 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:01.277952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:01.292923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:01.295035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:01.296419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:01.296419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:01.296419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:01.296419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:01.298436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 279/548] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:01.300125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:01.415382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69360465-09b2-4bd8-bbcc-3deffadc5327 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:01.458856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:01.460482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:01.460780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:01.461069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:01.461858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:01.462176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:01.463967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:01.478118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 280/549] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:01.483098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:01.502277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7274256-dbb1-4bb7-a496-e6b8e60b3388 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:01.511658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:01.538960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02090ba4-69e2-4fec-8958-9ca0c66e631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:01.540169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02090ba4-69e2-4fec-8958-9ca0c66e631a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:01.540690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02090ba4-69e2-4fec-8958-9ca0c66e631a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:01.541183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02090ba4-69e2-4fec-8958-9ca0c66e631a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:01.552500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02090ba4-69e2-4fec-8958-9ca0c66e631a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-02090ba4-69e2-4fec-8958-9ca0c66e631a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:01.560594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:01.561481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:01.562256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:01.564989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 270/550] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:01.574234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02090ba4-69e2-4fec-8958-9ca0c66e631a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:01.672675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb879ae-4065-4944-b35a-9a86fb3debcb None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:01.710487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:01.711702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:01.712214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:01.712667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:01.721070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:01.722763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:01.741239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:01.742394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:01.743507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:01.750094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 281/551] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:01.750936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:01.759182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:01.830102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:01.840375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02090ba4-69e2-4fec-8958-9ca0c66e631a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:01.862907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3f36-64eb-453b-8fc2-1694d38773fc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:01.899121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:01.900116 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:01.900942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:01.901394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:01.902474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:01.902957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:01.904711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:01.923556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 282/552] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Jul 15 15:33:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:01.931057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:01.941118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:01.941804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:01.942096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:01.942378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:01.943134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:01.943476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:01.945387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:01.967359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 271/553] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:01.980892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78dd36-167a-480a-b1ee-df1daac2b57b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:02.074421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:02.082782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317700a1-25f5-494b-8c24-753716700e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:02.083770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317700a1-25f5-494b-8c24-753716700e09 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.089542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317700a1-25f5-494b-8c24-753716700e09 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.089542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317700a1-25f5-494b-8c24-753716700e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.158070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317700a1-25f5-494b-8c24-753716700e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:02.159936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317700a1-25f5-494b-8c24-753716700e09 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:02.164312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83123abe-8894-4aae-b2be-ebf6fdc283ec None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:02.210765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff876a8-f604-465d-9d68-34db0dd22bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:02.212172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff876a8-f604-465d-9d68-34db0dd22bb0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.212651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff876a8-f604-465d-9d68-34db0dd22bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.213366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff876a8-f604-465d-9d68-34db0dd22bb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.216396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:02.231828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff876a8-f604-465d-9d68-34db0dd22bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:02.231828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff876a8-f604-465d-9d68-34db0dd22bb0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-5ff876a8-f604-465d-9d68-34db0dd22bb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:02.256833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:02.257949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:02.259087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:02.259087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:02.259087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:02.259087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:02.260799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 272/554] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:02.261904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d910f89-7234-4953-8d61-ca6484b432a9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.262196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d910f89-7234-4953-8d61-ca6484b432a9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.262470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d910f89-7234-4953-8d61-ca6484b432a9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.264043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 273/555] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:02.265833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d910f89-7234-4953-8d61-ca6484b432a9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.285022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 283/556] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:02.286604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e2d7d-ca9c-4320-b6e4-a11fb2119072 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.287097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e2d7d-ca9c-4320-b6e4-a11fb2119072 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.288218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e2d7d-ca9c-4320-b6e4-a11fb2119072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.305172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5e8e2d7d-ca9c-4320-b6e4-a11fb2119072 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:02.306652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5e8e2d7d-ca9c-4320-b6e4-a11fb2119072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:02.330763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:02.381731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d910f89-7234-4953-8d61-ca6484b432a9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:02.385895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 284/557] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:02.387020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e2d7d-ca9c-4320-b6e4-a11fb2119072 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.437285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:02.438035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.438226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.438511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.439236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:02.439577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:02.440980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:02.453783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 274/558] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:02.454724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.506240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e2d7d-ca9c-4320-b6e4-a11fb2119072 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:02.543684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:02.544556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:02.545525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.545987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.546411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.558237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:02.560094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:02.565740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf502ff-7149-4314-a507-293e7e99aea5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:02.579886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:02.581104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:02.582056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:02.582056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:02.582056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:02.582056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:02.584024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 285/559] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:02.585618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.598841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:02.606082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c3951-a801-4fdd-95e3-220e2fc132a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:02.613675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c3951-a801-4fdd-95e3-220e2fc132a2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.613675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c3951-a801-4fdd-95e3-220e2fc132a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.613675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c3951-a801-4fdd-95e3-220e2fc132a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.621664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c3951-a801-4fdd-95e3-220e2fc132a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e47c3951-a801-4fdd-95e3-220e2fc132a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:02.631735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:02.632683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:02.633551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:02.638435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 275/560] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:02.639342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c3951-a801-4fdd-95e3-220e2fc132a2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.709663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055c7df-a177-4734-a14a-67c55057eb86 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:02.747814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:02.748742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.749028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.749307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.757689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:02.759210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:02.760794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c3951-a801-4fdd-95e3-220e2fc132a2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:02.767953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:02.775178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:02.781080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 286/561] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:33:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:02.781993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.801271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48928d-c4ce-4df7-9803-9c69cb66ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:02.801938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48928d-c4ce-4df7-9803-9c69cb66ee89 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.802673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48928d-c4ce-4df7-9803-9c69cb66ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.802992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48928d-c4ce-4df7-9803-9c69cb66ee89 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.804297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48928d-c4ce-4df7-9803-9c69cb66ee89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:02.804610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48928d-c4ce-4df7-9803-9c69cb66ee89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:02.805991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48928d-c4ce-4df7-9803-9c69cb66ee89 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:02.825466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 276/562] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:02.825466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9101b-22e2-4b6d-9e40-3786e6535510 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.825466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9101b-22e2-4b6d-9e40-3786e6535510 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.825466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9101b-22e2-4b6d-9e40-3786e6535510 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.832919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 277/563] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:02.833752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9101b-22e2-4b6d-9e40-3786e6535510 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.861147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:02.894028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464be22-0cd3-4284-b2b2-892ff0a0ec7a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:02.930786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:02.931948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:02.932251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:02.932535 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:02.933444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:02.933759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:02.935152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:02.948096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 287/564] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:02.948096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:02.949238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9101b-22e2-4b6d-9e40-3786e6535510 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.033386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1eb3c3-a797-435a-8144-383904c5ab60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.034535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eb3c3-a797-435a-8144-383904c5ab60 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.034912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eb3c3-a797-435a-8144-383904c5ab60 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.035196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1eb3c3-a797-435a-8144-383904c5ab60 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.036058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1eb3c3-a797-435a-8144-383904c5ab60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:03.036366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1eb3c3-a797-435a-8144-383904c5ab60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.038376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1eb3c3-a797-435a-8144-383904c5ab60 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.082180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 278/565] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:33:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:03.082180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296cc83-4ca2-4cf0-be31-bc0af9767984 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.082180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296cc83-4ca2-4cf0-be31-bc0af9767984 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.082180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296cc83-4ca2-4cf0-be31-bc0af9767984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.098862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9296cc83-4ca2-4cf0-be31-bc0af9767984 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:03.100880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9296cc83-4ca2-4cf0-be31-bc0af9767984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:03.103500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf514b0-ed1b-4256-b429-685f8cd57cdc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.119077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:03.134124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:03.139075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.139918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.140208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.140493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.147876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.149280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:03.164268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:03.165371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 288/566] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:03.166250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.176761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 279/567] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:03.177605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296cc83-4ca2-4cf0-be31-bc0af9767984 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.270128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88969001-759c-49b9-80a8-e02c449d7721 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.276992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296cc83-4ca2-4cf0-be31-bc0af9767984 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.304236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6a4f-f243-451d-9689-63763aebbb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.304957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6a4f-f243-451d-9689-63763aebbb4b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.305239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6a4f-f243-451d-9689-63763aebbb4b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.305514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6a4f-f243-451d-9689-63763aebbb4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.310390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.311172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.311488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.311770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.318191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb6a4f-f243-451d-9689-63763aebbb4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.320206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb6a4f-f243-451d-9689-63763aebbb4b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.326458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.327976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-22fb6a4f-f243-451d-9689-63763aebbb4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:03.338533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:03.339619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:03.340493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:03.340493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:03.340493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:03.340493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:03.340493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 289/568] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:03.341045 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c393d05-b27b-4074-8a36-29bc676a4e0c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.341203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c393d05-b27b-4074-8a36-29bc676a4e0c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.341478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c393d05-b27b-4074-8a36-29bc676a4e0c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.343440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 290/569] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:03.344397 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c393d05-b27b-4074-8a36-29bc676a4e0c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:03.345853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:03.346857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:03.347988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:03.347988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:03.347988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:03.347988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:03.347988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 280/570] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:03.348365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.357603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:03.394275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:03.450626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c393d05-b27b-4074-8a36-29bc676a4e0c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.452698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c9da8-9f54-453d-a27b-a0ca8b894175 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.484172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.484828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.485113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.485325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ebb1f-7f7a-4f2f-8f05-0efaf58177c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.485447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.486113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ebb1f-7f7a-4f2f-8f05-0efaf58177c2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.486206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:03.486402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ebb1f-7f7a-4f2f-8f05-0efaf58177c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.486490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.486684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ebb1f-7f7a-4f2f-8f05-0efaf58177c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.487940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.497735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ebb1f-7f7a-4f2f-8f05-0efaf58177c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:03.499724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 291/571] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:03.500505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d38ebb1f-7f7a-4f2f-8f05-0efaf58177c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:03.505430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:03.506586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:03.507530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 281/572] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:03.508194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d35ab-1539-4914-a9e8-db77bc54ec5a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.508194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d35ab-1539-4914-a9e8-db77bc54ec5a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.508408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d35ab-1539-4914-a9e8-db77bc54ec5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.524823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cc9d35ab-1539-4914-a9e8-db77bc54ec5a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:03.526287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cc9d35ab-1539-4914-a9e8-db77bc54ec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:03.585538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:03.592581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 282/573] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:03.593236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d35ab-1539-4914-a9e8-db77bc54ec5a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.602019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963db3f9-da04-4e0e-b3ba-a7d6b2d58927 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.605441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:03.639265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.639956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.640246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.640539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.641258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:03.641567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.642945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.654697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 292/574] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:03.655510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.692828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d35ab-1539-4914-a9e8-db77bc54ec5a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.727900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.728563 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.728845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.729118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.729831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:03.730136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.731552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.743002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 283/575] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:03.744284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.755763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6f00-f19f-4de4-aad6-a55ae4c46a57 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.790659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.791340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.791616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.791914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.792727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=8f69205a5fc042299a8913c924b45e45) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:03.793037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.794461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.799123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:03.799992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 293/576] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/role_assignments?user.id=8f69205a5fc042299a8913c924b45e45&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:03.800648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.832545 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:03.853780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0ef19-2595-46ff-bbe5-0bb44d864fe2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.892495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.893191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.893478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.893756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.894518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:03.894836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:03.896323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:03.906077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 284/577] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:03.907173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:03.924976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9eb34b-1234-4eb7-9c6d-2d8991c7c3c1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:03.964507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f542d0-abd6-49e4-a05e-c890d26275e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:03.964802 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f542d0-abd6-49e4-a05e-c890d26275e2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:03.965093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f542d0-abd6-49e4-a05e-c890d26275e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:03.965374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f542d0-abd6-49e4-a05e-c890d26275e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:03.986379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f542d0-abd6-49e4-a05e-c890d26275e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a2f542d0-abd6-49e4-a05e-c890d26275e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:03.994238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:03.995183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:03.996078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 294/578] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:03.996754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f542d0-abd6-49e4-a05e-c890d26275e2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.027740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31291714-0154-439c-b5f1-7f211558a26a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.046387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:04.063412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468e677-a924-493f-9933-53defe4223dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.064251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468e677-a924-493f-9933-53defe4223dc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.064542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468e677-a924-493f-9933-53defe4223dc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.064818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468e677-a924-493f-9933-53defe4223dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.073727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468e677-a924-493f-9933-53defe4223dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.075188 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468e677-a924-493f-9933-53defe4223dc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b468e677-a924-493f-9933-53defe4223dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:04.090426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:04.091539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:33:04.092500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:04.096434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 285/579] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:04.097159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468e677-a924-493f-9933-53defe4223dc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.099730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:04.113350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f542d0-abd6-49e4-a05e-c890d26275e2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.149731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.151037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.151542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.151824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.158906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.160344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:04.174080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:04.175130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:04.176072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 295/580] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:04.176572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.209289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468e677-a924-493f-9933-53defe4223dc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.249338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.250131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.250413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.250686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.262576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.264150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:04.281886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:04.282771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:04.284007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:04.284007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:04.284007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:04.284007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:04.284007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 286/581] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:04.284007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.297031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cd79d-3f9e-4592-be74-659e4a98cedd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.316892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:04.333660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.335145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.335455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.335727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.336465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:04.336774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.347779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.347779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 296/582] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:04.347779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.364325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:04.403043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc1a47-7e53-4228-a9b7-92dfe22606ea None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.441625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.442297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.442586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.442868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.443845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:04.444109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.445550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.457539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29cb65-ec3c-43be-b370-1d0bfd5de7f8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.467885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 287/583] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:04.467885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.497128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.497791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.498073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.498348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.499126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:04.499481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.500927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.514674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 297/584] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:04.516654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.562282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:04.575438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67b28-2bc1-4e18-965f-7c25a339db18 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.593400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:04.616233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.616825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.617106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.617395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.618113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:04.618435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.619847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.626179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ea96b-c68c-4614-94db-a79e4bf16596 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.631451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 288/585] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:04.632348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.665417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.666217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.666505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.666789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.676982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.678982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:04.698022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:04.698920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:04.699814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:04.699814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:04.699814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:04.699814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:04.699814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 298/586] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:04.700087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.751836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971bf7-3b70-483e-96af-418a97c525b7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.787785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.788573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.788854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.789129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.799871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.801352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.804753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38488b-4098-4bad-b7a8-344fadb8b79e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.810407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:04.820513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:04.825757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:04.826993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:04.826993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:04.826993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:04.826993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:04.826993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 289/587] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:04.826993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:04.832474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:04.844372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e0436-f4c8-47b0-bb22-acaf38743ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.845553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e0436-f4c8-47b0-bb22-acaf38743ab4 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.845666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e0436-f4c8-47b0-bb22-acaf38743ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.845885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e0436-f4c8-47b0-bb22-acaf38743ab4 None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/bf670978cc2e4c529093a191bc26f885/roles/75570e166e6e4bd680698565f2d2cc0b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.861203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e0436-f4c8-47b0-bb22-acaf38743ab4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=bf670978cc2e4c529093a191bc26f885, role_id=75570e166e6e4bd680698565f2d2cc0b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.862717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e0436-f4c8-47b0-bb22-acaf38743ab4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.932180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6d01a-a290-417a-90c1-611cbe4cee40 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:04.966510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:04.967309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:04.967613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:04.967892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:04.975325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:04.976778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:04.990639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:04.991580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:04.992947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 290/588] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:04.993543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.007600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 299/589] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:04 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/bf670978cc2e4c529093a191bc26f885/roles/75570e166e6e4bd680698565f2d2cc0b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:33:05.009560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e0436-f4c8-47b0-bb22-acaf38743ab4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.097409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:05.156959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b16c78-32df-4b9d-811b-b7dc9e337e86 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.199902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.200641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.201027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.201404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.201966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e0436-f4c8-47b0-bb22-acaf38743ab4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.202338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:05.202745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.204246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.216700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 291/590] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:05.217663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.243342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.244490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.245200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.245633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.246515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:05.246989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.249016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.265154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 300/591] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:05.266295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.311714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:05.349590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e27cb-01ea-4ceb-b518-7d49ce1d5394 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.372788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:05.388302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.388976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.389469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.389577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.390594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:05.390967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.392463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.415354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 292/592] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:05.415354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.415854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d745d9e-2947-4281-8479-4ac76044623c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.463899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.464885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.465270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.466061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.476824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.478738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:05.501624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:05.502631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:33:05.503581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:05.507438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 301/593] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:05.509846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.532650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5ecd2-efe6-4715-9022-8f5c25796efe None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.575320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.576492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.577006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.577468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.585315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.587156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:05.602699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:05.603651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:05.604543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:05.604543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:05.604543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:05.604543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:05.604543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:05.604543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:05.604543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:05.604543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:05.606717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 293/594] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:05.607857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.634037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:05.645358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206c2b-a8d2-4c72-b43a-52416a0b1bf4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.654664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:05.688687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.689556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.689836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.690177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.704639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.706844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.730821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10beab3b-da30-45d9-aa9f-4b66b889d1e5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:05.739730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:05.741212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:05.742062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:05.742062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:05.742062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:05.742062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:05.744889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 302/595] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:05.745811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.783905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.784954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.785463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.785940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.787958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:05.788533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.790584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.818939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 294/596] 10.222.0.98 () {56 vars in 1139 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:05.818939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.855569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096b24f-5120-4cc3-a7e1-e23b01f730dc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.892646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.893693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.894276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.894656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.895477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:05.895792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.897908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.910851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 303/597] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:05.910851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:05.918644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:05.926330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:05.933953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe0a84-2599-4824-9ab1-6d64a762a494 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:05.968444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299627d2-ca89-43e9-97da-5f380a019072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:05.969105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299627d2-ca89-43e9-97da-5f380a019072 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.969393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299627d2-ca89-43e9-97da-5f380a019072 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.969675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299627d2-ca89-43e9-97da-5f380a019072 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.970390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299627d2-ca89-43e9-97da-5f380a019072 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:05.970698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299627d2-ca89-43e9-97da-5f380a019072 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:05.972184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299627d2-ca89-43e9-97da-5f380a019072 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:05.983555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 295/598] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:05.984771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00fe0-757c-4909-aaf1-946a8a30a0b2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:05.985088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00fe0-757c-4909-aaf1-946a8a30a0b2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:05.985376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba00fe0-757c-4909-aaf1-946a8a30a0b2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:05.991329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 296/599] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:05.991329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba00fe0-757c-4909-aaf1-946a8a30a0b2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:06.028206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24e41f-3c8b-410f-80fa-23902f72da2e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.072216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.073129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.073521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.073701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.084195 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:06.085703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:06.103115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:06.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:06.105189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:06.105189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:06.105189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:06.105189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:06.105189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 304/600] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:06.105523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:06.118729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba00fe0-757c-4909-aaf1-946a8a30a0b2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.154088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6f783-467b-4893-b9f9-0c0158c55a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.154982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f783-467b-4893-b9f9-0c0158c55a6a None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.155271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f783-467b-4893-b9f9-0c0158c55a6a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.155706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6f783-467b-4893-b9f9-0c0158c55a6a None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/52f18d0f5f634d84a99fc3e46963a8b4/roles/75570e166e6e4bd680698565f2d2cc0b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.169285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:06.174212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6f783-467b-4893-b9f9-0c0158c55a6a None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=52f18d0f5f634d84a99fc3e46963a8b4, role_id=75570e166e6e4bd680698565f2d2cc0b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:06.175780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6f783-467b-4893-b9f9-0c0158c55a6a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:06.212098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69b1db-091d-4092-8414-06dfe72cde9f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.245205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71013ad6-bf4c-428c-ad3f-9935c15040f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.245930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71013ad6-bf4c-428c-ad3f-9935c15040f0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.246212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71013ad6-bf4c-428c-ad3f-9935c15040f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.246486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71013ad6-bf4c-428c-ad3f-9935c15040f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.253001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71013ad6-bf4c-428c-ad3f-9935c15040f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:06.254445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71013ad6-bf4c-428c-ad3f-9935c15040f0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-71013ad6-bf4c-428c-ad3f-9935c15040f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:06.267845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:06.268884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:06.269748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:06.269748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:06.269748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:06.269748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:06.269748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:06.269748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:06.269748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:06.269748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:06.271431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 305/601] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:06.273373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78450e8a-4c6d-47ad-8461-efb7e7d6ff8c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.273671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78450e8a-4c6d-47ad-8461-efb7e7d6ff8c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.273956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78450e8a-4c6d-47ad-8461-efb7e7d6ff8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.286647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 297/602] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:05 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/52f18d0f5f634d84a99fc3e46963a8b4/roles/75570e166e6e4bd680698565f2d2cc0b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:33:06.288323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6f783-467b-4893-b9f9-0c0158c55a6a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:06.298325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:06.304518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-78450e8a-4c6d-47ad-8461-efb7e7d6ff8c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:06.305777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-78450e8a-4c6d-47ad-8461-efb7e7d6ff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:06.419342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 306/603] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:06.419342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78450e8a-4c6d-47ad-8461-efb7e7d6ff8c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:06.466884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6f783-467b-4893-b9f9-0c0158c55a6a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.477842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:06.516111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.517164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.517666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.518677 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.519657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:06.520600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:06.523186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:06.541221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 298/604] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:06.542338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:06.547845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78450e8a-4c6d-47ad-8461-efb7e7d6ff8c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.591864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.592823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.593215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.593600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.594670 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:06.595103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:06.597361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:06.613227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 307/605] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:06.614261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:06.615453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:06.662947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6267a4-806c-4bbe-aceb-e4a2bd9d30b5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.700552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae6784-748b-44f0-a945-e00bad55821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.701647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae6784-748b-44f0-a945-e00bad55821a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.702077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae6784-748b-44f0-a945-e00bad55821a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.702524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae6784-748b-44f0-a945-e00bad55821a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.718452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae6784-748b-44f0-a945-e00bad55821a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-22ae6784-748b-44f0-a945-e00bad55821a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:06.727087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:06.728253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:06.729124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:06.737013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 299/606] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:06.738387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae6784-748b-44f0-a945-e00bad55821a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:06.752545 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6e2ae-fd29-4604-ba74-ca09de25defe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.764747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:06.804389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.805132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.805425 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.805713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.814852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:06.816833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:06.835180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:06.836096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:06.836958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:06.836958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:06.836958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:06.836958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:06.836958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:06.836958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:06.836958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:06.836958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:06.838057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 308/607] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:06.838057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:06.866942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae6784-748b-44f0-a945-e00bad55821a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.895847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:06.906366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ea682-e969-4e03-bdd6-92ecd1a072bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.909066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ea682-e969-4e03-bdd6-92ecd1a072bc None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.909066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ea682-e969-4e03-bdd6-92ecd1a072bc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.909066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ea682-e969-4e03-bdd6-92ecd1a072bc None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/68015ac9febe42ef917ca58fae7b1c26/roles/75570e166e6e4bd680698565f2d2cc0b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.926023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ea682-e969-4e03-bdd6-92ecd1a072bc None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=68015ac9febe42ef917ca58fae7b1c26, role_id=75570e166e6e4bd680698565f2d2cc0b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:06.927864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ea682-e969-4e03-bdd6-92ecd1a072bc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:06.957314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a92ce4-0700-4b60-a937-b16e151a268a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:06.994367 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:06.995051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:06.995359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:06.995640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:06.996392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:06.996702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:06.998108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:07.011587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 309/608] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:07.011587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.040233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 300/609] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:06 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/68015ac9febe42ef917ca58fae7b1c26/roles/75570e166e6e4bd680698565f2d2cc0b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:33:07.041936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ea682-e969-4e03-bdd6-92ecd1a072bc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.051195 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:07.184208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec19d-d566-4b84-9953-3a4d52bef612 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:07.202351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ea682-e969-4e03-bdd6-92ecd1a072bc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:07.226497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4f09b-7cba-4d2d-bd75-531607385c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:07.227565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f09b-7cba-4d2d-bd75-531607385c3e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.228120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f09b-7cba-4d2d-bd75-531607385c3e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.228609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f09b-7cba-4d2d-bd75-531607385c3e None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.229593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4f09b-7cba-4d2d-bd75-531607385c3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:07.230113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4f09b-7cba-4d2d-bd75-531607385c3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:07.232673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4f09b-7cba-4d2d-bd75-531607385c3e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:07.242387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9ff3d-a514-4e8d-a77d-d9ffc9fc7c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:07.243185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9ff3d-a514-4e8d-a77d-d9ffc9fc7c9e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.243519 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9ff3d-a514-4e8d-a77d-d9ffc9fc7c9e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.243800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9ff3d-a514-4e8d-a77d-d9ffc9fc7c9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.247667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 310/610] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:07.249039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b999-92ae-4117-aed2-caf38fc49566 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.249332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b999-92ae-4117-aed2-caf38fc49566 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.249614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7b999-92ae-4117-aed2-caf38fc49566 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.250861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 311/611] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:07.253814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7b999-92ae-4117-aed2-caf38fc49566 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.256888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9ff3d-a514-4e8d-a77d-d9ffc9fc7c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:07.258389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9ff3d-a514-4e8d-a77d-d9ffc9fc7c9e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b2d9ff3d-a514-4e8d-a77d-d9ffc9fc7c9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:07.279214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:07.280236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:07.281102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:07.281102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:07.281102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:07.281102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:07.281102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 301/612] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:07.282055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eac43-0f6c-46fc-99bd-b1fb59d048e8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.282345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eac43-0f6c-46fc-99bd-b1fb59d048e8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.282629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1eac43-0f6c-46fc-99bd-b1fb59d048e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.305273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cb1eac43-0f6c-46fc-99bd-b1fb59d048e8 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:07.306708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cb1eac43-0f6c-46fc-99bd-b1fb59d048e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:07.330364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:07.367152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:07.383320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 302/613] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:07.383320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1eac43-0f6c-46fc-99bd-b1fb59d048e8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.389572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7b999-92ae-4117-aed2-caf38fc49566 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:07.444397 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f094d5a-d5dc-447d-ad5f-7e23819bfd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:07.445693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f094d5a-d5dc-447d-ad5f-7e23819bfd7c None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.446124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f094d5a-d5dc-447d-ad5f-7e23819bfd7c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.446644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f094d5a-d5dc-447d-ad5f-7e23819bfd7c None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/9d20de9c921942d1afeb63e87272f782/roles/75570e166e6e4bd680698565f2d2cc0b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.463053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f094d5a-d5dc-447d-ad5f-7e23819bfd7c None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=9d20de9c921942d1afeb63e87272f782, role_id=75570e166e6e4bd680698565f2d2cc0b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:07.464805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f094d5a-d5dc-447d-ad5f-7e23819bfd7c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:07.491318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1eac43-0f6c-46fc-99bd-b1fb59d048e8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:07.528769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:07.529681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.530151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.530589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.531501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:07.531956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:07.533541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:07.547633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 312/614] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:07 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/9d20de9c921942d1afeb63e87272f782/roles/75570e166e6e4bd680698565f2d2cc0b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:33:07.548920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f094d5a-d5dc-447d-ad5f-7e23819bfd7c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.549574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 303/615] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:07.555459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.578651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:07.666913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e446de-2f13-4f49-b7e0-bb9d3cc0b2e6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:07.707432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:07.708225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.708516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.708795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.729333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:07.730814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:07.736527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f094d5a-d5dc-447d-ad5f-7e23819bfd7c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:07.767685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:07.768703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:07.769648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 304/616] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:07.777487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee2f66-e0e2-43bd-90c3-ed1100c201de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:07.778589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2f66-e0e2-43bd-90c3-ed1100c201de None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.778824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2f66-e0e2-43bd-90c3-ed1100c201de None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.779275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2f66-e0e2-43bd-90c3-ed1100c201de None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.780320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.788839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:07.793879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee2f66-e0e2-43bd-90c3-ed1100c201de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-1eee2f66-e0e2-43bd-90c3-ed1100c201de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:07.802276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:07.808066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:07.808894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 313/617] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:07.810271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee2f66-e0e2-43bd-90c3-ed1100c201de None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.906957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:07.923517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b606d-daf5-497b-a8fb-4f3faf6e2316 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:07.935375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee2f66-e0e2-43bd-90c3-ed1100c201de None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:07.965513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:07.965513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.965513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.965513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.966111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:07.966111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:07.966111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:07.978193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:07.979564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:07.979925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:07.980426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:07.981363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:07.981761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:07.982751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 305/618] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:07.983366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:07.988279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:07.997671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 314/619] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:08.000097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:08.077694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:08.097054 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dde2b-7f4e-417c-b4e5-221ac9d0109e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:08.154498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96715e-e0e0-4e41-a4ed-ca45bcbea1d0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:08.159267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:08.160197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.160593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.160941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.168392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:08.183039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:08.184555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:08.213735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:08.215097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:08.216211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:08.216211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:08.216211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:08.216211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:08.217493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 306/620] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:08.225267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:08.226074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.226366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.226651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.229388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:08.249738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:08.251401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:08.285475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:08.286415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:08.287277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:08.287277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:08.287277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:08.287277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:08.287277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 315/621] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:08.293904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:08.419712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721ae99-3b2d-4d60-a294-74497d21692d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:08.434929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:08.458969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:08.460380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.460669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.460952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.461943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:08.462253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:08.463669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:08.471201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918fc2f-7ce2-4e83-b56d-ba9cab38350a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:08.477542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 316/622] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:08.484804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:08.549328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:08.550111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.550472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.550761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.551511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:08.551825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:08.553749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:08.566708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 307/623] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:08.572161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:08.626545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:08.632708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e047c-7dc8-4107-a144-f0ae5b93c550 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:08.676290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f277f16-4d52-41d6-9c6d-a24225e94fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:08.677625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f277f16-4d52-41d6-9c6d-a24225e94fd7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.678340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f277f16-4d52-41d6-9c6d-a24225e94fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.679171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f277f16-4d52-41d6-9c6d-a24225e94fd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.688346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b40f07-2e3d-46f4-88ff-ab7fb4067ec0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:08.688710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f277f16-4d52-41d6-9c6d-a24225e94fd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:08.690563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f277f16-4d52-41d6-9c6d-a24225e94fd7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-7f277f16-4d52-41d6-9c6d-a24225e94fd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:08.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:08.711636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:08.712872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:08.715406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 317/624] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:33:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:08.716770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177420e-6ea4-47b3-b0c6-e3a36c9f8d1b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.717174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177420e-6ea4-47b3-b0c6-e3a36c9f8d1b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.717328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177420e-6ea4-47b3-b0c6-e3a36c9f8d1b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.722655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 318/625] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:08.722655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177420e-6ea4-47b3-b0c6-e3a36c9f8d1b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:08.732500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238063c4-7a60-4c64-ba81-7898a6c0ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:08.733292 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238063c4-7a60-4c64-ba81-7898a6c0ab0e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.733573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238063c4-7a60-4c64-ba81-7898a6c0ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.733854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238063c4-7a60-4c64-ba81-7898a6c0ab0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.742542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238063c4-7a60-4c64-ba81-7898a6c0ab0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:08.744043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238063c4-7a60-4c64-ba81-7898a6c0ab0e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:08.758681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-238063c4-7a60-4c64-ba81-7898a6c0ab0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:08.762393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:08.763256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:08.764311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:08.764311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:08.764311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:08.764311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:08.764311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:08.764311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:08.764311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:08.764311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:08.767372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 308/626] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:08.768935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fe174-5cc0-4521-a9f1-9bd29e11a10b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.769229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fe174-5cc0-4521-a9f1-9bd29e11a10b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.769512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fe174-5cc0-4521-a9f1-9bd29e11a10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.788249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-032fe174-5cc0-4521-a9f1-9bd29e11a10b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:08.789688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-032fe174-5cc0-4521-a9f1-9bd29e11a10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:08.869745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177420e-6ea4-47b3-b0c6-e3a36c9f8d1b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:08.879313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 309/627] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:08.879313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fe174-5cc0-4521-a9f1-9bd29e11a10b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:08.920783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:08.928971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:08.929933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:08.930452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:08.930964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:08.932054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:08.932561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:08.934213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:08.952625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 319/628] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:33:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:08.953926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:08.997710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fe174-5cc0-4521-a9f1-9bd29e11a10b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.035748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:09.036918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.037610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.038198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.039246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:09.039791 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:09.041655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:09.060218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 310/629] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:09.061497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:09.064172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:09.074003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c8fc-4b9e-4d9f-9c37-9b1d969ffbe7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.112645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a78435-3f9c-4450-80cd-e966837ad8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:09.113444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78435-3f9c-4450-80cd-e966837ad8aa None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.113733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78435-3f9c-4450-80cd-e966837ad8aa None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.114008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a78435-3f9c-4450-80cd-e966837ad8aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.125605 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a78435-3f9c-4450-80cd-e966837ad8aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d0a78435-3f9c-4450-80cd-e966837ad8aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:09.133944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:09.134845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:09.135678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:09.140048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 320/630] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:09.140970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a78435-3f9c-4450-80cd-e966837ad8aa None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:09.173662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:09.178849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129348dd-76ff-4a2d-96ab-f922a2a6355d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.216877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a50e9-e92c-4b37-926f-e8b56a1a3397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:09.218361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a50e9-e92c-4b37-926f-e8b56a1a3397 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.218863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a50e9-e92c-4b37-926f-e8b56a1a3397 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.219385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a50e9-e92c-4b37-926f-e8b56a1a3397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.253509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a78435-3f9c-4450-80cd-e966837ad8aa None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.289953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d41c5c-6687-4d8a-a1e6-9057932a83ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:09.290577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d41c5c-6687-4d8a-a1e6-9057932a83ef None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.290872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d41c5c-6687-4d8a-a1e6-9057932a83ef None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.291151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d41c5c-6687-4d8a-a1e6-9057932a83ef None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/8f69205a5fc042299a8913c924b45e45/roles/75570e166e6e4bd680698565f2d2cc0b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.293246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a50e9-e92c-4b37-926f-e8b56a1a3397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:09.295087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a50e9-e92c-4b37-926f-e8b56a1a3397 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:09.307102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d41c5c-6687-4d8a-a1e6-9057932a83ef None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=8f69205a5fc042299a8913c924b45e45, role_id=75570e166e6e4bd680698565f2d2cc0b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:09.308725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d41c5c-6687-4d8a-a1e6-9057932a83ef None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:09.384773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:09.399386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 321/631] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:09 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/8f69205a5fc042299a8913c924b45e45/roles/75570e166e6e4bd680698565f2d2cc0b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:33:09.399386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d41c5c-6687-4d8a-a1e6-9057932a83ef None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:09.406702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:09.424587 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 311/632] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:09.426493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cae2a-5001-4ae3-89c9-059ff849501a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.426790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cae2a-5001-4ae3-89c9-059ff849501a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.427779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cae2a-5001-4ae3-89c9-059ff849501a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.430423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 312/633] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:09.431254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cae2a-5001-4ae3-89c9-059ff849501a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:09.519223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d41c5c-6687-4d8a-a1e6-9057932a83ef None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.564656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577b0f6-f46f-42d2-9854-19abe7a66c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:09.565667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b0f6-f46f-42d2-9854-19abe7a66c18 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.565960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b0f6-f46f-42d2-9854-19abe7a66c18 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.566245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b0f6-f46f-42d2-9854-19abe7a66c18 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.567016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577b0f6-f46f-42d2-9854-19abe7a66c18 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:09.567399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577b0f6-f46f-42d2-9854-19abe7a66c18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:09.568259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cae2a-5001-4ae3-89c9-059ff849501a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.569461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577b0f6-f46f-42d2-9854-19abe7a66c18 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:09.580785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 322/634] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:09.582658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db19e6-d7f7-407b-8521-0e21e12d841d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.583007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db19e6-d7f7-407b-8521-0e21e12d841d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.583317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db19e6-d7f7-407b-8521-0e21e12d841d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.602746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-21db19e6-d7f7-407b-8521-0e21e12d841d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:09.604037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-21db19e6-d7f7-407b-8521-0e21e12d841d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:09.620495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:09.621284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.621569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.621845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.632974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:09.634438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:09.653008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:09.654100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:09.654991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:09.654991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:09.654991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:09.654991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:09.654991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 313/635] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:09.655263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:09.671253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:09.681483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 323/636] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:09.683803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db19e6-d7f7-407b-8521-0e21e12d841d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:09.726918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:09.778427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c09bc-6012-4a3d-8d6f-0cc606f620a4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.810986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db19e6-d7f7-407b-8521-0e21e12d841d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.813891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:09.815095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.815406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.815684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.826477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:09.827962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:09.846365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:09.847346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:09.848297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:09.848297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:09.848297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:09.848297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:09.848297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 314/637] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:09.848860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:09.855316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:09.855316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:09.855316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:09.855606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:09.857053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:09.857373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:09.858814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:09.878799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 324/638] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:09.879868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:09.950207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:09.984839 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822bb4b-111f-461d-90c6-df95223c6254 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:09.986767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:10.001275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5c641-fea4-4089-a74d-6f0b422a0d0f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.020358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e269e36-d6b2-4a0d-a6b2-8a32b7858ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.021391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e269e36-d6b2-4a0d-a6b2-8a32b7858ecf None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.021674 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e269e36-d6b2-4a0d-a6b2-8a32b7858ecf None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.023493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e269e36-d6b2-4a0d-a6b2-8a32b7858ecf None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.033647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e269e36-d6b2-4a0d-a6b2-8a32b7858ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:10.039712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e4d56-8405-46dd-9248-3840a7b19199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.040370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4d56-8405-46dd-9248-3840a7b19199 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.040654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4d56-8405-46dd-9248-3840a7b19199 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.040929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e4d56-8405-46dd-9248-3840a7b19199 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.041643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e4d56-8405-46dd-9248-3840a7b19199 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:10.041949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e4d56-8405-46dd-9248-3840a7b19199 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7e269e36-d6b2-4a0d-a6b2-8a32b7858ecf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:10.042414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:10.043274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:10.044125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:10.044746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e4d56-8405-46dd-9248-3840a7b19199 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.046671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 315/639] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:10.051532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e269e36-d6b2-4a0d-a6b2-8a32b7858ecf None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.061927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 325/640] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:10.061927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dd7f4-086a-40e9-b7da-fded767b0f84 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.061927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dd7f4-086a-40e9-b7da-fded767b0f84 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.061927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dd7f4-086a-40e9-b7da-fded767b0f84 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.061927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 326/641] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:10.062335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dd7f4-086a-40e9-b7da-fded767b0f84 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.167898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e269e36-d6b2-4a0d-a6b2-8a32b7858ecf None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.171090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dd7f4-086a-40e9-b7da-fded767b0f84 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.185158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:10.204084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd11ee7-d657-41e2-9139-8ca85c4164c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.204868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11ee7-d657-41e2-9139-8ca85c4164c6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.205152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11ee7-d657-41e2-9139-8ca85c4164c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.205434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11ee7-d657-41e2-9139-8ca85c4164c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.206746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.207466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.207750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.208026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.208768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:10.212199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.212199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.213575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd11ee7-d657-41e2-9139-8ca85c4164c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.215019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd11ee7-d657-41e2-9139-8ca85c4164c6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.224553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 327/642] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:10.224553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-4fd11ee7-d657-41e2-9139-8ca85c4164c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:10.230643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:10.232092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:10.232968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:10.232968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:10.232968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:10.232968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:10.232968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:10.232968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:10.232968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:10.232968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:10.236749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 316/643] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:10.238013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab40b91-800f-48bd-99fb-c06e9b07cc0c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.238301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab40b91-800f-48bd-99fb-c06e9b07cc0c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.238572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab40b91-800f-48bd-99fb-c06e9b07cc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.258497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0ab40b91-800f-48bd-99fb-c06e9b07cc0c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:10.260641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0ab40b91-800f-48bd-99fb-c06e9b07cc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:10.287030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:10.342978 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640143d7-4376-416c-90d7-6c77b6f56962 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.355326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 317/644] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:10.355326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab40b91-800f-48bd-99fb-c06e9b07cc0c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.390021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.390914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.391202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.391519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] PATH_INFO: `/v3/users/glance` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.399791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.401512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:10.415607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:10.416540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 328/645] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:10.417589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.456252 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:10.479088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab40b91-800f-48bd-99fb-c06e9b07cc0c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.513959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.514752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.515041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.515349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.525715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981ddb5-3c1f-463d-9f9b-42da5289fa0b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.526322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.527819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:10.545761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:10.546799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:10.547696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:10.547696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:10.547696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:10.547696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:10.547696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 318/646] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:10.547696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.552884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:10.566239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.567009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.567284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.567612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.568390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:10.568720 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.574075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.589536 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 329/647] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:10.592546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.667877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e21467-d373-4a3f-bac7-6654b41a0bcb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.695820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:10.703221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf7bd7-d81b-43b8-a3f0-622821039472 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.705804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63907f80-22fd-407e-bd4a-2ddf6d1682bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.709919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63907f80-22fd-407e-bd4a-2ddf6d1682bb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.709919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63907f80-22fd-407e-bd4a-2ddf6d1682bb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.709919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63907f80-22fd-407e-bd4a-2ddf6d1682bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.718055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63907f80-22fd-407e-bd4a-2ddf6d1682bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-63907f80-22fd-407e-bd4a-2ddf6d1682bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:10.725784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:10.726884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 319/648] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:10.727756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63907f80-22fd-407e-bd4a-2ddf6d1682bb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.745118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.745785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.746071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.746345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.747171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:10.747503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.748944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.767010 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 330/649] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:10.772189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.800385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:10.837974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63907f80-22fd-407e-bd4a-2ddf6d1682bb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.873595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.874253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.874537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.874813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.875206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a05e5-9ffa-4fb0-ae9e-650b97efac3c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:10.875559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:10.875870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.877463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.890395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 320/650] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:10.891197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.920446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:10.922709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:10.922709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:10.922709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:10.922709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=bf670978cc2e4c529093a191bc26f885) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:10.922986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:10.924581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:10.939328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 331/651] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/role_assignments?user.id=bf670978cc2e4c529093a191bc26f885&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:10.939328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:10.961803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.029192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4b19d-ebec-44b0-9d8c-6f012db67b29 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.062375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f1fbf-518a-4c9f-b27c-c83a71c46d4d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.062821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.077809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:11.078916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.079583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.079964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.081275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:11.081589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:11.083321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:11.096831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 321/652] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:11.097046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:11.114567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7dc1-51a3-4cb7-9337-21f0180bb4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:11.115351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7dc1-51a3-4cb7-9337-21f0180bb4c3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.115745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7dc1-51a3-4cb7-9337-21f0180bb4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.116088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7dc1-51a3-4cb7-9337-21f0180bb4c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.208344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd7dc1-51a3-4cb7-9337-21f0180bb4c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:11.210474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd7dc1-51a3-4cb7-9337-21f0180bb4c3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:11.223229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc444a20-6252-450a-9c4b-3fc57f2e2173 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.229974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.269867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:11.271167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.271508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.271960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.283734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:11.286263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:11.304854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:11.305999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:11.307267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 322/653] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:11.308142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:11.323323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 332/654] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:11.328455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7dc1-51a3-4cb7-9337-21f0180bb4c3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:11.367282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.445346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7dc1-51a3-4cb7-9337-21f0180bb4c3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.451652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f826dab-99c7-45f1-ae0d-4776a2f8011e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.463559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.481905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:11.482912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.483391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.483847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.484849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:11.485318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:11.486926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:11.497828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:11.498637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.498926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.499209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.511248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 333/655] 10.222.0.98 () {56 vars in 1137 bytes} [Fri Jul 15 15:33:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:11.511974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:11.512605 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:11.513571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:11.535788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:11.536738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:11.537642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:11.537642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:11.537642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:11.537642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:11.539370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 323/656] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:11.540343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:11.637324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bc577-eab0-4495-8587-03e95a906c03 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.646886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.651788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440f950-794e-40fb-a441-f89f9bcbb029 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.674157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:11.674951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.677813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.677813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.685285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f3a8f-5dc3-490c-8238-74949e9da328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:11.685936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f3a8f-5dc3-490c-8238-74949e9da328 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.686223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f3a8f-5dc3-490c-8238-74949e9da328 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.686502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f3a8f-5dc3-490c-8238-74949e9da328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.686611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:11.688023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:11.707373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:11.708347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:11.709234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:11.709234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:11.709234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:11.709234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:11.713690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 334/657] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:11.714540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:11.741635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.760478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f3a8f-5dc3-490c-8238-74949e9da328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:11.761932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f3a8f-5dc3-490c-8238-74949e9da328 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:11.822864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5d55e-6e13-46a6-b089-cf7354eb6721 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.858478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:11.859437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.859727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.860163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.861196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:11.861511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:11.862898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:11.866306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.870915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 324/658] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:11.872079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d59d3-1bd5-4419-b1e8-32e64b314be7 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:11.872426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d59d3-1bd5-4419-b1e8-32e64b314be7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:11.872657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d59d3-1bd5-4419-b1e8-32e64b314be7 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:11.875482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 325/659] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:11.875482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d59d3-1bd5-4419-b1e8-32e64b314be7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:11.882355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 335/660] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:11.883144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:11.971008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:11.985938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154ee3d-6811-40ac-8a4a-c0bb6f53a48a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:11.986477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d59d3-1bd5-4419-b1e8-32e64b314be7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.019804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.020597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.020881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.021151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.021686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2b773-f561-434e-9322-65b22f371f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.022893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2b773-f561-434e-9322-65b22f371f9f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.023665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2b773-f561-434e-9322-65b22f371f9f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.023665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2b773-f561-434e-9322-65b22f371f9f None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.024730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2b773-f561-434e-9322-65b22f371f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:12.025055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2b773-f561-434e-9322-65b22f371f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.026708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2b773-f561-434e-9322-65b22f371f9f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.032819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.034257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.044323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 326/661] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:12.044323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352cda-fe6d-49b8-a4ef-360069a67c01 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.044323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352cda-fe6d-49b8-a4ef-360069a67c01 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.044323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1352cda-fe6d-49b8-a4ef-360069a67c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:12.056648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:12.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:12.058383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:12.058383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:12.058383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:12.058383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:12.060180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e1352cda-fe6d-49b8-a4ef-360069a67c01 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:12.061374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e1352cda-fe6d-49b8-a4ef-360069a67c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:12.062340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 336/662] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:12.063266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.121109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:12.147389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 327/663] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:12.147389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1352cda-fe6d-49b8-a4ef-360069a67c01 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.172901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22797f-2c92-4cae-a329-6d62ccf52e81 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.216287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.217095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.217384 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.217662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.227040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.228947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.231251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:12.244398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:12.251161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 337/664] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:12.252216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.263952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1352cda-fe6d-49b8-a4ef-360069a67c01 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.299011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.299837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.300123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.300462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.307878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.309345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:12.323190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:12.324266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:12.325239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 328/665] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:12.325781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.355778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59ac18-11fe-4382-852f-db3110867f20 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.377363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:12.397185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.398067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.398514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.398793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.399549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:12.399862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.401566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.421867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 338/666] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:12.421867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.436290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e054f-1533-4df5-8488-7509f1ef8278 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.471803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be1a1-9c24-441d-8b90-b0c2d1b1914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.472594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be1a1-9c24-441d-8b90-b0c2d1b1914e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.472878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be1a1-9c24-441d-8b90-b0c2d1b1914e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.473150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be1a1-9c24-441d-8b90-b0c2d1b1914e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.482192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:12.484300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987be1a1-9c24-441d-8b90-b0c2d1b1914e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-987be1a1-9c24-441d-8b90-b0c2d1b1914e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:12.492109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:12.498938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 329/667] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:12.499780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be1a1-9c24-441d-8b90-b0c2d1b1914e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.530112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a756f-747d-48e1-a7e1-e7f4d8e60cbe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.565086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.565769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.566058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.566336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.567167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:12.567508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.569494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.588043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 339/668] 10.222.0.98 () {56 vars in 1139 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:12.593145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.606533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be1a1-9c24-441d-8b90-b0c2d1b1914e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.616429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:12.643417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.644073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.644359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.644637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.645414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:12.645723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.647217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.664615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 330/669] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:12.664615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.708071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670117fc-1718-47f4-a523-827f8f6f0b92 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.726407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:12.745231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.745969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.746260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.746541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.753422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.754945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:12.769609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:12.770518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:12.771433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:12.771433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:12.771433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:12.771433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:12.771433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:12.771433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:12.771433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:12.771433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:12.774906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644e576-19f1-4b21-9ddb-ba00d5b783ba None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.775441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 340/670] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:12.776949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.828207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.828900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.829411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.829542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.830210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:12.830516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.831964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:12.842533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 331/671] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:12.843265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:12.876754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:12.895393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3990138-4964-4622-8b7f-6a8ba9942bd5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.939042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94ee8c-a0bb-48a3-9bec-bef49e3ad48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.939731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94ee8c-a0bb-48a3-9bec-bef49e3ad48c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.940026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94ee8c-a0bb-48a3-9bec-bef49e3ad48c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.940314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94ee8c-a0bb-48a3-9bec-bef49e3ad48c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.951494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37426b30-6f91-4f3b-b2f4-516e7c22aa99 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:12.988170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:12.988512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:12.989803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:12.990236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:12.991178 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:12.992171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=52f18d0f5f634d84a99fc3e46963a8b4) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:12.992628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:12.994293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.011475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 332/672] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/role_assignments?user.id=52f18d0f5f634d84a99fc3e46963a8b4&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:13.011475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.025437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94ee8c-a0bb-48a3-9bec-bef49e3ad48c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:13.027809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94ee8c-a0bb-48a3-9bec-bef49e3ad48c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.129732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:13.137843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a0faf-2cc3-48fa-8293-0dbc92e23841 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:13.147448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 341/673] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:13.147448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94ee8c-a0bb-48a3-9bec-bef49e3ad48c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.185658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8265d-2a00-40eb-822b-c64ad930ffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:13.186332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8265d-2a00-40eb-822b-c64ad930ffdd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.186618 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8265d-2a00-40eb-822b-c64ad930ffdd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.186899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8265d-2a00-40eb-822b-c64ad930ffdd None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.187759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8265d-2a00-40eb-822b-c64ad930ffdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:13.188073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8265d-2a00-40eb-822b-c64ad930ffdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:13.189543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8265d-2a00-40eb-822b-c64ad930ffdd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.204359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 333/674] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:13.206122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fefcf-fb3e-4ec4-8659-b2233b9f3d88 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.206415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fefcf-fb3e-4ec4-8659-b2233b9f3d88 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.206693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fefcf-fb3e-4ec4-8659-b2233b9f3d88 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.208120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 334/675] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:13.208736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fefcf-fb3e-4ec4-8659-b2233b9f3d88 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.243193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:13.267481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94ee8c-a0bb-48a3-9bec-bef49e3ad48c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:13.305788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:13.306601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.306894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.307342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.318264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:13.319828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.323312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fefcf-fb3e-4ec4-8659-b2233b9f3d88 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:13.338595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:13.339699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:13.340744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:13.340744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:13.340744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:13.340744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:13.340744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 342/676] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:13.344705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.362190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c88723-5fc2-4957-b144-d7567cab8ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:13.362978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c88723-5fc2-4957-b144-d7567cab8ba0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.363265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c88723-5fc2-4957-b144-d7567cab8ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.363582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c88723-5fc2-4957-b144-d7567cab8ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.373894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c88723-5fc2-4957-b144-d7567cab8ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:13.375374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c88723-5fc2-4957-b144-d7567cab8ba0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-92c88723-5fc2-4957-b144-d7567cab8ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:13.395109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:13.396226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:13.397385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:13.397385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:13.397385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:13.397385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:13.397385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 335/677] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:13.397951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea54cf-e4ef-4710-91b8-08618c527c03 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.398239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea54cf-e4ef-4710-91b8-08618c527c03 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.398519 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea54cf-e4ef-4710-91b8-08618c527c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.411866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:13.419517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-95ea54cf-e4ef-4710-91b8-08618c527c03 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:13.420976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-95ea54cf-e4ef-4710-91b8-08618c527c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:13.468128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8d53-6ddd-4869-a81d-4208ae809ce6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:13.495489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:13.499338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 336/678] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:13.499338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea54cf-e4ef-4710-91b8-08618c527c03 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.515667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:13.516938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.517933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.518416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.519318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=68015ac9febe42ef917ca58fae7b1c26) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:13.519644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:13.521591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.529695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 343/679] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/role_assignments?user.id=68015ac9febe42ef917ca58fae7b1c26&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:13.539334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.650734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea54cf-e4ef-4710-91b8-08618c527c03 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:13.686377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab201abd-d7c0-4a96-b451-9da74b9d5366 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:13.694113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:13.694997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.695309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.695610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.696341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:13.696654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:13.698212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.701484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:13.717797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 337/680] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:13.718740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.731715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:13.732398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.732684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.732961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.733688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:13.733993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:13.737819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.759343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 344/681] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:13.759343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.821533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:13.838871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba40c76-1cc7-4c11-8217-e0e87e97b790 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:13.874140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b36d6-844e-4fb1-862d-27624ca51b49 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:13.882879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ece50c-46fd-44b0-9545-606fd4ead558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:13.883811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece50c-46fd-44b0-9545-606fd4ead558 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.884170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece50c-46fd-44b0-9545-606fd4ead558 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.884873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ece50c-46fd-44b0-9545-606fd4ead558 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.898210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ece50c-46fd-44b0-9545-606fd4ead558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e8ece50c-46fd-44b0-9545-606fd4ead558 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:13.908457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:13.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:13.910231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 338/682] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:13.910942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ece50c-46fd-44b0-9545-606fd4ead558 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.921920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:13.922722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.923004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.923284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.931455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:13.932899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:13.948671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:13.949721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:13.950624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:13.954612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 345/683] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:13.955395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:13.987168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:14.049279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ece50c-46fd-44b0-9545-606fd4ead558 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.070806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023be20e-b5e2-4831-8ca7-6c4f01f255d4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.086300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:14.095288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.099415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.099415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.099415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.105543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:14.107109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:14.113080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.114206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.114679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.115097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.116124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:14.116600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:14.118274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:14.124047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:14.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:14.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:14.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:14.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:14.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:14.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:14.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:14.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:14.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:14.134813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 339/685] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:14.134813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:14.135223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 346/685] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:14.136307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:14.245715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3be07-b368-4162-83ca-6466ba768667 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.252092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9ac84-2033-43a5-855b-7c9fcdc9c521 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.264399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:14.289075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.290235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.290843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.291441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.292716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:14.293217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.293339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:14.294042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.294402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.294755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.295575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:14.295680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:14.296103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:14.298661 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:14.316504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 340/686] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Jul 15 15:33:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:14.317664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:14.321219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 347/687] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:14.324054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:14.374446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:14.441758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6aadf-6f6d-4740-9a8a-243e6c13bd50 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.476803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.477627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.477914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.478195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.479633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91a69b-39b7-485a-9c91-37fcae3d2dcd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.488448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:14.489960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:14.510135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:14.511025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:14.511941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:14.511941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:14.511941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:14.511941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:14.515554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 341/688] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:14.516394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:14.527658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7657e9d-0e4b-4930-a71b-977bc7729658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.529149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7657e9d-0e4b-4930-a71b-977bc7729658 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.529650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7657e9d-0e4b-4930-a71b-977bc7729658 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.530159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7657e9d-0e4b-4930-a71b-977bc7729658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.546832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:14.619498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7657e9d-0e4b-4930-a71b-977bc7729658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:14.621090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7657e9d-0e4b-4930-a71b-977bc7729658 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:14.639187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfed25-c1c8-40a6-b304-0d09e91aaddd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.656102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:14.688840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.689951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.691348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.692200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.693505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=9d20de9c921942d1afeb63e87272f782) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:14.694524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:14.697311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:14.711828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 342/689] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:14 2022] GET /identity/v3/role_assignments?user.id=9d20de9c921942d1afeb63e87272f782&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:14.711828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:14.758869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 348/690] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:14.765098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7657e9d-0e4b-4930-a71b-977bc7729658 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:14.894208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:14.905931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15942b53-f90f-43ea-9a29-e805e571a48f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.940385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7657e9d-0e4b-4930-a71b-977bc7729658 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:14.957439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.958248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.958542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.958819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.959868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:14.960185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:14.962488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:14.977777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 343/691] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:14.983146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:14.993334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:14.994203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:14.994486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:14.994828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:14.996950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:15.006388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.008403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:15.028292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:15.029250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:15.030202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:15.030202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:15.030202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:15.030202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:15.034892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 349/692] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:15.035505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:15.119009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd74c70-ca0c-48db-8746-3a1036666edc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:15.174509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de61a04-7b64-46e3-a6f8-c8d07f7520ac None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:15.200082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9815a-e994-45ef-b214-070166c91842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:15.201411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9815a-e994-45ef-b214-070166c91842 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.201714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9815a-e994-45ef-b214-070166c91842 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.202278 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9815a-e994-45ef-b214-070166c91842 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.213284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9815a-e994-45ef-b214-070166c91842 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.213770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0f546-fa62-4943-b598-3cfa70cf7e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:15.214448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f546-fa62-4943-b598-3cfa70cf7e4a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.214737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f546-fa62-4943-b598-3cfa70cf7e4a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.214840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9815a-e994-45ef-b214-070166c91842 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:15.215020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0f546-fa62-4943-b598-3cfa70cf7e4a None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.215794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0f546-fa62-4943-b598-3cfa70cf7e4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:15.216106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0f546-fa62-4943-b598-3cfa70cf7e4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.218414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0f546-fa62-4943-b598-3cfa70cf7e4a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:15.230996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 350/693] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:15.232346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d219b-c580-4efc-a4be-400037598a33 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.232644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d219b-c580-4efc-a4be-400037598a33 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.232930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d219b-c580-4efc-a4be-400037598a33 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.234435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 351/694] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a7f9815a-e994-45ef-b214-070166c91842 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:15.235580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:15.236497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:15.237403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:15.238242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 344/695] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:33:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:15.241232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d219b-c580-4efc-a4be-400037598a33 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:15.242873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0e840-c43b-47ec-a98a-fc42a3eeebc5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.243162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0e840-c43b-47ec-a98a-fc42a3eeebc5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.243967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0e840-c43b-47ec-a98a-fc42a3eeebc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.263957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1af0e840-c43b-47ec-a98a-fc42a3eeebc5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:15.265162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1af0e840-c43b-47ec-a98a-fc42a3eeebc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:15.269087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:15.311664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:15.354655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 345/696] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:15.354655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0e840-c43b-47ec-a98a-fc42a3eeebc5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:15.390053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d219b-c580-4efc-a4be-400037598a33 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:15.430275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:15.431003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.431304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.431755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.439645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.441262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:15.457152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:15.458059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:15.458928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:15.458928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:15.458928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:15.458928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:15.458928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:15.458928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:15.458928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:15.458928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:15.463146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 352/697] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:15.464043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:15.490451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0e840-c43b-47ec-a98a-fc42a3eeebc5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:15.530232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:15.531122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.531599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.532267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.533347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:15.533671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.535208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:15.546646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:15.556304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 346/698] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:33:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:15.557236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:15.590426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faac7d0-751e-4bc0-835a-bfb18d5b349d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:15.601041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:15.637746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:15.638564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.638854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.639137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.652288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.653929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:15.670955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc5547-8b03-45a8-821f-21cdb8ad3048 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:15.676979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:15.677899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:15.678874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:15.678874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:15.678874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:15.678874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:15.683622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 353/699] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:15.684437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:15.713840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:15.714927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.715538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.716418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.717411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:15.717929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.719941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:15.738352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 347/700] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:15.739709 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:15.806144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72d85-dcab-4821-adb5-6ab4427c8457 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:15.810775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:15.845067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc1023-a762-4e55-8a54-3a85e3dda1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:15.845732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc1023-a762-4e55-8a54-3a85e3dda1c1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.846023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc1023-a762-4e55-8a54-3a85e3dda1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.846306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc1023-a762-4e55-8a54-3a85e3dda1c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.855699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26ade5-2bec-41ec-9ada-538a87cf0d8d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:15.873975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:15.891974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:15.892995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:15.893289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:15.893568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:15.894329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:15.894642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.896061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:15.913848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 348/701] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:15.913848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:15.914634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc1023-a762-4e55-8a54-3a85e3dda1c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:15.916490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc1023-a762-4e55-8a54-3a85e3dda1c1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.020031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbdcea-9226-4cf1-9c80-68889325b52e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:16.025036 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 354/702] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:16.026705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0136c-7bf6-4db2-9816-d2ba34dab992 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.026997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0136c-7bf6-4db2-9816-d2ba34dab992 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.027275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0136c-7bf6-4db2-9816-d2ba34dab992 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.028849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 355/703] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:16.029778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0136c-7bf6-4db2-9816-d2ba34dab992 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:16.043462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:16.061741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:16.062450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.062742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.063018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.063880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=8f69205a5fc042299a8913c924b45e45) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:16.064193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:16.065639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.071933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 349/704] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:15 2022] GET /identity/v3/role_assignments?user.id=8f69205a5fc042299a8913c924b45e45&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:16.072601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:16.115344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:16.180160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0136c-7bf6-4db2-9816-d2ba34dab992 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:16.195642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a7a6b-5ea1-41d8-9cc4-46eba1d3e567 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:16.236060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611a42e-19ab-4faf-bf0c-8d5940136037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:16.236738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611a42e-19ab-4faf-bf0c-8d5940136037 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.237752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611a42e-19ab-4faf-bf0c-8d5940136037 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.238060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611a42e-19ab-4faf-bf0c-8d5940136037 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.238768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611a42e-19ab-4faf-bf0c-8d5940136037 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:16.240279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611a42e-19ab-4faf-bf0c-8d5940136037 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.251836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563d3de-008c-4c10-b870-41b1335b8069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:16.252650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563d3de-008c-4c10-b870-41b1335b8069 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.252925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563d3de-008c-4c10-b870-41b1335b8069 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.253199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563d3de-008c-4c10-b870-41b1335b8069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.265066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563d3de-008c-4c10-b870-41b1335b8069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:16.269304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563d3de-008c-4c10-b870-41b1335b8069 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-9563d3de-008c-4c10-b870-41b1335b8069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:16.283846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:16.284749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:16.286014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:16.286014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:16.286014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:16.286014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:16.286014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 350/705] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:16.286493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a518692-02e3-4139-980e-705b8d6638f3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.286780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a518692-02e3-4139-980e-705b8d6638f3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.287063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a518692-02e3-4139-980e-705b8d6638f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.304386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0a518692-02e3-4139-980e-705b8d6638f3 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:16.305636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0a518692-02e3-4139-980e-705b8d6638f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:16.327053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 356/706] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:33:16 2022] POST /identity/v3/users => generated 254 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:16.327922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611a42e-19ab-4faf-bf0c-8d5940136037 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:16.337532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:16.392787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 351/707] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:16.455530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a518692-02e3-4139-980e-705b8d6638f3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:16.494637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611a42e-19ab-4faf-bf0c-8d5940136037 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:16.523776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:16.543596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:16.544249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.544543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.544833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.545570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:16.545882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:16.547985 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.564391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 357/708] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:16.585810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:16.631539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a518692-02e3-4139-980e-705b8d6638f3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:16.681410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:16.682526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.682821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.683107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.683960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:16.686190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:16.705345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:16.706337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:16.707160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:16.714797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 352/709] 10.222.0.98 () {56 vars in 1129 bytes} [Fri Jul 15 15:33:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:16.716921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:16.724017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654ece-e69f-4216-be26-23842df8f521 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:16.758529 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:16.764349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:16.765167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.765460 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.765746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.774305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:16.775856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:16.794064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:16.794947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:16.796274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 358/710] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:16.801260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:16.843770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:16.855891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56181c-3c42-4dd7-aafc-57c44cdb1488 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:16.909170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:16.909869 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.910154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.910435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.911218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:16.911561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:16.913664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.926957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 353/711] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Jul 15 15:33:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:16.933138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:16.936641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7261d-9037-46c2-808c-51e539788f04 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:16.971704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:16.972683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:16.973184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:16.973643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:16.975716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:16.976218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:16.977871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:16.993239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 359/712] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:17.003735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:17.078982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:17.088961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79952fc6-a88f-4f1e-9d93-afdcaa097b05 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:17.132092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7191f-1bc5-4823-9aed-c3599e578f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:17.133201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7191f-1bc5-4823-9aed-c3599e578f30 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:17.133773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7191f-1bc5-4823-9aed-c3599e578f30 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:17.134068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7191f-1bc5-4823-9aed-c3599e578f30 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:17.135019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7191f-1bc5-4823-9aed-c3599e578f30 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:17.135370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7191f-1bc5-4823-9aed-c3599e578f30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:17.137452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7191f-1bc5-4823-9aed-c3599e578f30 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:17.146256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 354/713] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Jul 15 15:33:16 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:17.147672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb79f-e394-4d8e-a48b-4a23c9e17a08 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:17.152668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb79f-e394-4d8e-a48b-4a23c9e17a08 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:17.152961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfb79f-e394-4d8e-a48b-4a23c9e17a08 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:17.159185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 355/714] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:17.166028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff4568-98ed-4f36-83af-547f5eed2888 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:17.166343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff4568-98ed-4f36-83af-547f5eed2888 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:17.166586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff4568-98ed-4f36-83af-547f5eed2888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:17.195479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:17.200542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-91ff4568-98ed-4f36-83af-547f5eed2888 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:17.201775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-91ff4568-98ed-4f36-83af-547f5eed2888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:17.210513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a587ce-2ac9-4b65-b5ce-be33d345dc22 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:17.279111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:17.279815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:17.280095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:17.280371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:17.285686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=bf670978cc2e4c529093a191bc26f885) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:17.286006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:17.287507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:17.303157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 360/715] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:16 2022] GET /identity/v3/role_assignments?user.id=bf670978cc2e4c529093a191bc26f885&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:17.361105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 356/716] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:33:17.387404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:17.514156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f6547-9d38-4cac-9067-f951d6aaeb77 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:17.556488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:17.557300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:17.557582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:17.557858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] PATH_INFO: `/v3/services/network` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:17.558516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:17.560040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:17.577024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:17.578050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:17.579111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 361/717] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:17 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:17.586017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff4568-98ed-4f36-83af-547f5eed2888 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:17.771759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:17.787347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff4568-98ed-4f36-83af-547f5eed2888 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:17.826965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e783b8f-0090-4e0d-9714-f2da2ca951f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:17.827664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e783b8f-0090-4e0d-9714-f2da2ca951f3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:17.827950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e783b8f-0090-4e0d-9714-f2da2ca951f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:17.828222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e783b8f-0090-4e0d-9714-f2da2ca951f3 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:17.828935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e783b8f-0090-4e0d-9714-f2da2ca951f3 None admin] RBAC: Adding query filter params (name=network) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:17.829232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e783b8f-0090-4e0d-9714-f2da2ca951f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:17.830611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e783b8f-0090-4e0d-9714-f2da2ca951f3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:17.839155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 357/718] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:17.845053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:17.893193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:18.006057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262fe47-51d3-488a-a011-7b49aa7017e0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:18.068141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ab441-49aa-4f21-9129-7cf66b908938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:18.069783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab441-49aa-4f21-9129-7cf66b908938 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:18.070286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab441-49aa-4f21-9129-7cf66b908938 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:18.070766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab441-49aa-4f21-9129-7cf66b908938 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:18.071719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ab441-49aa-4f21-9129-7cf66b908938 None admin] RBAC: Adding query filter params (type=network) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:18.072234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ab441-49aa-4f21-9129-7cf66b908938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:18.073993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ab441-49aa-4f21-9129-7cf66b908938 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:18.081005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 362/719] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:18.394240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d1980-a8e8-4551-83ec-55b42a31a3b2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:18.394897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d1980-a8e8-4551-83ec-55b42a31a3b2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:18.395418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d1980-a8e8-4551-83ec-55b42a31a3b2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:18.397320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 358/720] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:18.402559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19ed8-aed5-4f39-9860-65dae5c803fc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:18.403454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19ed8-aed5-4f39-9860-65dae5c803fc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:18.403948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19ed8-aed5-4f39-9860-65dae5c803fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:18.424773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a1d19ed8-aed5-4f39-9860-65dae5c803fc None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:18.425999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a1d19ed8-aed5-4f39-9860-65dae5c803fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:18.515621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 363/721] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:18.550959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d1980-a8e8-4551-83ec-55b42a31a3b2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:18.685070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d1980-a8e8-4551-83ec-55b42a31a3b2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:18.732918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4910e9-2aff-4cf0-95b0-e44b06d689d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:18.733723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4910e9-2aff-4cf0-95b0-e44b06d689d2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:18.734013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4910e9-2aff-4cf0-95b0-e44b06d689d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:18.734294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4910e9-2aff-4cf0-95b0-e44b06d689d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:18.749609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4910e9-2aff-4cf0-95b0-e44b06d689d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0b4910e9-2aff-4cf0-95b0-e44b06d689d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:18.759208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:18.760175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:18.760992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:18.765666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 359/722] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:18.767053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19ed8-aed5-4f39-9860-65dae5c803fc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:18.786054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:18.899367 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19ed8-aed5-4f39-9860-65dae5c803fc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:18.942494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99aa37b-7ae9-47ed-b07c-5a6a96ad8574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:18.943099 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa37b-7ae9-47ed-b07c-5a6a96ad8574 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:18.943410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa37b-7ae9-47ed-b07c-5a6a96ad8574 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:18.943692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99aa37b-7ae9-47ed-b07c-5a6a96ad8574 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:18.944582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99aa37b-7ae9-47ed-b07c-5a6a96ad8574 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:18.944902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99aa37b-7ae9-47ed-b07c-5a6a96ad8574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:18.946291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99aa37b-7ae9-47ed-b07c-5a6a96ad8574 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:18.955423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 364/723] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:18.961377 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4910e9-2aff-4cf0-95b0-e44b06d689d2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:18.992808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:19.090504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49548a7-a23b-4677-983a-bd1b21b54102 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.090857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49548a7-a23b-4677-983a-bd1b21b54102 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.091073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49548a7-a23b-4677-983a-bd1b21b54102 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.092481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 365/724] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:19.098591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0568da-ecea-4dc1-91be-571bbed2538a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.098860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0568da-ecea-4dc1-91be-571bbed2538a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.099147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0568da-ecea-4dc1-91be-571bbed2538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.115032 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4910e9-2aff-4cf0-95b0-e44b06d689d2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:19.132738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-eb0568da-ecea-4dc1-91be-571bbed2538a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:19.133936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-eb0568da-ecea-4dc1-91be-571bbed2538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:19.193676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:19.194433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.194719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.194996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.219227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 366/725] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:33:19.221358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:19.222847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:19.244956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:19.245902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:19.246892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:19.246892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:19.246892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:19.246892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:19.246892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 360/726] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:19.251794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0568da-ecea-4dc1-91be-571bbed2538a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:19.252381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:19.269007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:19.389945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:19.434595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0568da-ecea-4dc1-91be-571bbed2538a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:19.455138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62a3b-0988-4847-bd23-78b236e19b40 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:19.503212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:19.503913 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.504245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.504550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.505340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:19.505649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:19.507614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:19.516576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2bb9d-a7f6-45a7-b142-baea1ff2aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:19.517370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2bb9d-a7f6-45a7-b142-baea1ff2aa1e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.517658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2bb9d-a7f6-45a7-b142-baea1ff2aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.517938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2bb9d-a7f6-45a7-b142-baea1ff2aa1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.532335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 367/727] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:19.535251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2bb9d-a7f6-45a7-b142-baea1ff2aa1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:19.536434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b4f2bb9d-a7f6-45a7-b142-baea1ff2aa1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:19.548392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:19.549363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:19.550200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 361/728] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:19.556245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2bb9d-a7f6-45a7-b142-baea1ff2aa1e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:19.658688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:19.715449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7dddc-987f-4411-a1c3-b45ebccef17d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:19.745814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2bb9d-a7f6-45a7-b142-baea1ff2aa1e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:19.770026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d71447-c991-4fa0-9416-682213d92db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:19.770834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71447-c991-4fa0-9416-682213d92db4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.771121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71447-c991-4fa0-9416-682213d92db4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.771445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71447-c991-4fa0-9416-682213d92db4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.776799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:19.784267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d71447-c991-4fa0-9416-682213d92db4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:19.786124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d71447-c991-4fa0-9416-682213d92db4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:19.794228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:19.794228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.794228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.794228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.794569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:19.794569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:19.794569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-e7d71447-c991-4fa0-9416-682213d92db4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:19.802470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:19.803937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:19.804866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 368/729] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:19.805920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78017047-f27e-46c6-b7af-cbd6c21ec710 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.806342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78017047-f27e-46c6-b7af-cbd6c21ec710 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.806532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78017047-f27e-46c6-b7af-cbd6c21ec710 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.806898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 362/730] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:19.811350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 369/731] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:19.811490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:19.816307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91eb47-c0b9-4fa0-a95d-690eb6b92562 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.816600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91eb47-c0b9-4fa0-a95d-690eb6b92562 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.816885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91eb47-c0b9-4fa0-a95d-690eb6b92562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.838477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5b91eb47-c0b9-4fa0-a95d-690eb6b92562 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:19.840325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5b91eb47-c0b9-4fa0-a95d-690eb6b92562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:19.912123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250ca08-aaa3-4acb-a5cd-379435f861d0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:19.921595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 370/732] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:19.922440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91eb47-c0b9-4fa0-a95d-690eb6b92562 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:19.955106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed73959-15cc-4203-beac-77f4ddd54272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:19.956164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73959-15cc-4203-beac-77f4ddd54272 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:19.956651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73959-15cc-4203-beac-77f4ddd54272 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:19.957132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73959-15cc-4203-beac-77f4ddd54272 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:19.958140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed73959-15cc-4203-beac-77f4ddd54272 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:19.958639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed73959-15cc-4203-beac-77f4ddd54272 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:19.960253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed73959-15cc-4203-beac-77f4ddd54272 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:20.059282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 363/733] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Jul 15 15:33:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:20.060612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c808b6-f91f-4901-81a7-b4dce38fee1c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.060904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c808b6-f91f-4901-81a7-b4dce38fee1c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.061180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c808b6-f91f-4901-81a7-b4dce38fee1c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.062606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 364/734] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:20.064024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c808b6-f91f-4901-81a7-b4dce38fee1c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:20.083607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:20.110799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:20.127234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91eb47-c0b9-4fa0-a95d-690eb6b92562 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.166594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.167575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.167866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.168180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.173970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c808b6-f91f-4901-81a7-b4dce38fee1c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.180471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:20.181958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:20.199986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:20.200880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:20.201733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:20.201733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:20.201733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:20.201733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:20.205888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 371/735] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:20.206559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:20.213896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebe4e5-3a9e-4465-b2f6-604195d291fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.214692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebe4e5-3a9e-4465-b2f6-604195d291fc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.214972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebe4e5-3a9e-4465-b2f6-604195d291fc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.215243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebe4e5-3a9e-4465-b2f6-604195d291fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.227129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebe4e5-3a9e-4465-b2f6-604195d291fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-66ebe4e5-3a9e-4465-b2f6-604195d291fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:20.234644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:20.235868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:20.236692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 365/736] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:20.237566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90028-1ab1-4c7c-8ec3-a2dac9a9be98 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.237801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90028-1ab1-4c7c-8ec3-a2dac9a9be98 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.238075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90028-1ab1-4c7c-8ec3-a2dac9a9be98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.255682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ceb90028-1ab1-4c7c-8ec3-a2dac9a9be98 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:20.257412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ceb90028-1ab1-4c7c-8ec3-a2dac9a9be98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:20.317715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a573b5-e7f3-48b8-b89d-ef00972f1b51 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.329132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:20.330001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 366/737] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:20.330874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90028-1ab1-4c7c-8ec3-a2dac9a9be98 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:20.362417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:20.364543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6daf1-d18b-4f86-827e-a733da2e6095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.365198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6daf1-d18b-4f86-827e-a733da2e6095 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.365642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6daf1-d18b-4f86-827e-a733da2e6095 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.366604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6daf1-d18b-4f86-827e-a733da2e6095 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.440122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6daf1-d18b-4f86-827e-a733da2e6095 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:20.440735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90028-1ab1-4c7c-8ec3-a2dac9a9be98 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.441602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6daf1-d18b-4f86-827e-a733da2e6095 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:20.476355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.477121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.477396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.477679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.478395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:20.478704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:20.480099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:20.491954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 367/738] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:20.496900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:20.545729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 372/739] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:20.546557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6daf1-d18b-4f86-827e-a733da2e6095 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:20.561458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:20.597840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:20.607016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e13053-74da-4cc5-a276-2c573c967b9a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.641080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.641740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.642023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.642296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.643025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:20.643362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:20.644724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:20.657727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 368/740] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:20.657727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:20.664156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6daf1-d18b-4f86-827e-a733da2e6095 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.698995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275fab0-174d-43be-a36a-e8ee1cea59b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.699707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275fab0-174d-43be-a36a-e8ee1cea59b3 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.700003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275fab0-174d-43be-a36a-e8ee1cea59b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.700278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275fab0-174d-43be-a36a-e8ee1cea59b3 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.700992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275fab0-174d-43be-a36a-e8ee1cea59b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:20.702392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275fab0-174d-43be-a36a-e8ee1cea59b3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:20.757470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9930b-dff8-46fc-be77-1a01b721dacc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.794330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.795164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.795426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 373/741] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:33:20 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:20.795559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.795719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.796261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275fab0-174d-43be-a36a-e8ee1cea59b3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:20.815356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:20.817139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:20.834644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:20.836399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 369/742] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:33:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:20.837337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:20.858125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:20.886695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:20.932857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275fab0-174d-43be-a36a-e8ee1cea59b3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.953603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1c137-f7a5-4cd7-b9b6-3f9140cc9d20 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:20.974392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91a277-9db0-4f3d-826c-a6d4e83b0d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.975259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a277-9db0-4f3d-826c-a6d4e83b0d11 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.975971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a277-9db0-4f3d-826c-a6d4e83b0d11 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.975971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a277-9db0-4f3d-826c-a6d4e83b0d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:20.988781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91a277-9db0-4f3d-826c-a6d4e83b0d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:20.990356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91a277-9db0-4f3d-826c-a6d4e83b0d11 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:20.995130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:20.996221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:20.996814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:20.997270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.009840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-dc91a277-9db0-4f3d-826c-a6d4e83b0d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:21.010323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:21.011393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:21.012286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:21.012286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:21.012286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:21.012286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:21.012286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 374/743] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:21.013720 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7646-3044-412d-b0ee-62ec30a3f194 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.014211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7646-3044-412d-b0ee-62ec30a3f194 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.014552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7646-3044-412d-b0ee-62ec30a3f194 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.016105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 375/744] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:21.016835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f7646-3044-412d-b0ee-62ec30a3f194 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.017027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:21.039055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:21.040246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:21.041318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:21.041318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:21.041318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:21.041318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:21.043314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 370/745] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:21.044361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.113503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:21.140921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f7646-3044-412d-b0ee-62ec30a3f194 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.153182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4053b-6dfd-463b-8a70-8fefe073d017 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.159747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:21.176624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a1d19-4ee4-48a7-b917-f6ef8f629121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:21.177461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a1d19-4ee4-48a7-b917-f6ef8f629121 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.177847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a1d19-4ee4-48a7-b917-f6ef8f629121 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.178808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a1d19-4ee4-48a7-b917-f6ef8f629121 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.179864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a1d19-4ee4-48a7-b917-f6ef8f629121 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:21.180279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a1d19-4ee4-48a7-b917-f6ef8f629121 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.181827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a1d19-4ee4-48a7-b917-f6ef8f629121 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.190776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:21.191483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.191779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.192056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.192777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:21.193095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.194526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.201335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 376/746] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:21.202679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e716e-1412-40ba-8c5f-32d546460fb5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.203068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e716e-1412-40ba-8c5f-32d546460fb5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.203475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e716e-1412-40ba-8c5f-32d546460fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.216196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 371/747] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:21.216880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.221453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f34e716e-1412-40ba-8c5f-32d546460fb5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:21.222766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f34e716e-1412-40ba-8c5f-32d546460fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:21.303804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 377/748] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:33:21.303804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e716e-1412-40ba-8c5f-32d546460fb5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.336662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cc30e-ab2f-4b9c-b16d-3f912ffadb50 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.369703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:21.382334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:21.382334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.382334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.382334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.384226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:21.384226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.384226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.394961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 372/749] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:21.394961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.404607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e716e-1412-40ba-8c5f-32d546460fb5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.411949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:21.438264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267821a-c166-4335-80d0-32e9d6b1e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:21.438923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267821a-c166-4335-80d0-32e9d6b1e590 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.439209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267821a-c166-4335-80d0-32e9d6b1e590 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.439542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267821a-c166-4335-80d0-32e9d6b1e590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.505605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc2952-fb58-4944-97e2-70d84dfaf32d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.511752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267821a-c166-4335-80d0-32e9d6b1e590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.513339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267821a-c166-4335-80d0-32e9d6b1e590 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.542917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:21.543722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.544179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.544306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.551124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.552767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:21.567666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:21.568778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 373/750] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:21.569648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.620570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:21.627163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 378/751] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:33:21.627163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267821a-c166-4335-80d0-32e9d6b1e590 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.632058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:21.679065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace14860-a7a1-423d-82d1-313c23bf3099 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.714668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f77b6-9857-40e1-80c0-8c753da0b289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:21.715339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f77b6-9857-40e1-80c0-8c753da0b289 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.715632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f77b6-9857-40e1-80c0-8c753da0b289 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.715912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f77b6-9857-40e1-80c0-8c753da0b289 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.716634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f77b6-9857-40e1-80c0-8c753da0b289 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.717995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f77b6-9857-40e1-80c0-8c753da0b289 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.722302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267821a-c166-4335-80d0-32e9d6b1e590 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.754870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:21.755692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.755981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.756285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.764166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.765614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:21.779916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:21.781014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:21.781981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 379/752] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:21.782805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.829790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 374/753] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:33:21 2022] POST /identity/v3/services => generated 240 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:21.830613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f77b6-9857-40e1-80c0-8c753da0b289 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:21.859678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:21.903981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab66e34-8175-4684-a6e3-27cd3af7801f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.953304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:21.954403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:21.955002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:21.955627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:21.956713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:21.959368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:21.959368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:21.961008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f77b6-9857-40e1-80c0-8c753da0b289 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:21.963320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:21.979401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 380/754] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:21.979401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.008224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:22.009273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:22.009714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:22.009993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:22.021595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:22.023835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:22.045699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:22.046802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:22.047919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:22.047919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:22.047919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:22.047919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:22.047919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 375/755] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:22.048251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.128144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d955f0e-f3aa-4103-a69d-63d1738a3ff8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:22.171735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:22.172153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c653f75-6d19-4bc3-a4ee-1ade9366d69a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:22.175891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:22.176678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:22.177065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:22.177444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:22.178439 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:22.179389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:22.180950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:22.199259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 381/756] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:33:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:22.203499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.217967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:22.218997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:22.219531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:22.220010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:22.221182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=8f69205a5fc042299a8913c924b45e45) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:22.221697 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:22.223450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:22.230408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 376/757] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:22 2022] GET /identity/v3/role_assignments?user.id=8f69205a5fc042299a8913c924b45e45&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:22.231493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.270361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:22.333348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df043e-4d83-49c4-934a-280a21cb27b6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:22.426268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5e212-4803-4bde-87c4-c0240368b635 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:22.429037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:22.429898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:22.430318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:22.430727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:22.434545 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:22.435064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:22.436712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:22.452618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 382/758] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:22.459476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.471864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:22.472101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:22.472342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:22.472617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:22.486013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:22.491383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:22.508308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:22.509406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:22.510501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:22.510501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:22.510501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:22.510501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:22.510501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 377/759] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:22.515472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.551650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:22.593108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:22.627437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace703f8-1c1c-4397-abb0-9928a19878b2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:22.666563 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:22.667231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:22.667546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:22.667825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:22.668995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:22.669312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:22.670873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:22.684525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 378/760] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:22.690353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.726664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48a931-7d0f-4403-ae64-f488ccafad89 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:22.805276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b5dfa-2587-47da-bd2b-c61a284ce63d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:22.837017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:22.842846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:22.855536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:22.855536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:22.855714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:22.855803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:22.855887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:22.855887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:22.855887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:22.856074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:22.856074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:22.867994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:22.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:22.869771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 379/761] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:22.870852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:22.872597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:22.875157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:22.908847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:22.909791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:22.910677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:22.910677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:22.910677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:22.910677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:22.910677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:22.910677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:22.910677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:22.910677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:22.913795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 383/762] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:22.924348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:22.992658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691c1ac-cd1e-4f3c-80b9-0176de2901e5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:23.041950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf9cf3-7cb0-4fe1-abc0-cd5ecd971efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:23.042893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9cf3-7cb0-4fe1-abc0-cd5ecd971efb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.043186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9cf3-7cb0-4fe1-abc0-cd5ecd971efb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.043501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf9cf3-7cb0-4fe1-abc0-cd5ecd971efb None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.044541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf9cf3-7cb0-4fe1-abc0-cd5ecd971efb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:23.044860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf9cf3-7cb0-4fe1-abc0-cd5ecd971efb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:23.045888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:23.048074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf9cf3-7cb0-4fe1-abc0-cd5ecd971efb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:23.049188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b46a0-06e0-4596-87b7-3df97fcf3f5c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:23.067326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 380/763] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:23.067326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf618f-702e-47bc-9964-5e2264cb04d8 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.067326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf618f-702e-47bc-9964-5e2264cb04d8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.067326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf618f-702e-47bc-9964-5e2264cb04d8 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.071408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 381/764] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:23.072398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf618f-702e-47bc-9964-5e2264cb04d8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:23.102880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914cf5d-c5a5-4ed5-a09a-e5c813c91257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:23.103964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cf5d-c5a5-4ed5-a09a-e5c813c91257 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.104582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cf5d-c5a5-4ed5-a09a-e5c813c91257 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.105154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914cf5d-c5a5-4ed5-a09a-e5c813c91257 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.106150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cf5d-c5a5-4ed5-a09a-e5c813c91257 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:23.106660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cf5d-c5a5-4ed5-a09a-e5c813c91257 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:23.108639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914cf5d-c5a5-4ed5-a09a-e5c813c91257 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:23.122632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 384/765] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:23.124654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cdec3-3bfa-4017-bb09-4b2feda53565 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.125192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cdec3-3bfa-4017-bb09-4b2feda53565 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.125698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cdec3-3bfa-4017-bb09-4b2feda53565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.135928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:23.146733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0f3cdec3-3bfa-4017-bb09-4b2feda53565 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:23.148338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0f3cdec3-3bfa-4017-bb09-4b2feda53565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:23.208549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf618f-702e-47bc-9964-5e2264cb04d8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:23.230872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 385/766] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:23.232066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cdec3-3bfa-4017-bb09-4b2feda53565 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:23.265547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9bc55-e171-4404-8d60-bc7489d67ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:23.266866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9bc55-e171-4404-8d60-bc7489d67ea6 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.267381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9bc55-e171-4404-8d60-bc7489d67ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.267880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9bc55-e171-4404-8d60-bc7489d67ea6 None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/bf670978cc2e4c529093a191bc26f885/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.270119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9bc55-e171-4404-8d60-bc7489d67ea6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=bf670978cc2e4c529093a191bc26f885, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:23.271879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9bc55-e171-4404-8d60-bc7489d67ea6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:23.339011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:23.342469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 382/767] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:23 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/bf670978cc2e4c529093a191bc26f885/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:33:23.344505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9bc55-e171-4404-8d60-bc7489d67ea6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:23.349947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cdec3-3bfa-4017-bb09-4b2feda53565 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:23.414240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:23.414924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.415216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.419677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.420549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=a1daa10e8ac5455cbf98f02971639237) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:23.420865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:23.422645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:23.436538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 386/768] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:23 2022] GET /identity/v3/role_assignments?user.id=a1daa10e8ac5455cbf98f02971639237&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:23.471994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9bc55-e171-4404-8d60-bc7489d67ea6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:23.484997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:23.512273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ee023-b216-497f-9714-02001a428918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:23.513134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ee023-b216-497f-9714-02001a428918 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.513424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ee023-b216-497f-9714-02001a428918 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.513814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ee023-b216-497f-9714-02001a428918 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.514727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ee023-b216-497f-9714-02001a428918 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:23.516423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ee023-b216-497f-9714-02001a428918 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-828ee023-b216-497f-9714-02001a428918 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:23.531356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:23.534648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:23.535505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 383/769] 10.222.0.98 () {56 vars in 1129 bytes} [Fri Jul 15 15:33:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:23.539187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:23.681160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a01b4-f423-4df9-b75f-31278a3f158f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:23.727425 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:23.728104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.728393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.728677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.729388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:23.729699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:23.731086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:23.741957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 387/770] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Jul 15 15:33:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:23.747691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ee023-b216-497f-9714-02001a428918 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:23.885424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ee023-b216-497f-9714-02001a428918 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:23.926916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:23.927946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:23.928478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:23.929426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:23.930334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:23.930819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:23.932503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:23.940879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 384/771] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Jul 15 15:33:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:23.946835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:23.974632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:24.075863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaa3d6-8853-440f-bf5d-ddd174c5b56a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:24.119343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979f6a1-484b-472c-a23b-62d835b12a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:24.120036 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f6a1-484b-472c-a23b-62d835b12a21 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.120326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f6a1-484b-472c-a23b-62d835b12a21 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.120606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979f6a1-484b-472c-a23b-62d835b12a21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.121408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979f6a1-484b-472c-a23b-62d835b12a21 None admin] RBAC: Adding query filter params (interface=public, service_id=124bc5f275c3442c991c8c2306a22b21, region_id=RegionOne) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:24.121723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979f6a1-484b-472c-a23b-62d835b12a21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:24.123326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979f6a1-484b-472c-a23b-62d835b12a21 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:24.132046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 388/772] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Jul 15 15:33:23 2022] GET /identity/v3/endpoints?service_id=124bc5f275c3442c991c8c2306a22b21&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:24.137195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:24.179919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c106c-393a-4405-ad72-b31c7dcff8b1 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.180251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c106c-393a-4405-ad72-b31c7dcff8b1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.180601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c106c-393a-4405-ad72-b31c7dcff8b1 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.182284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 389/773] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:24.188392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3c80-d30e-41ba-a1c3-023017f8f651 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.188685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3c80-d30e-41ba-a1c3-023017f8f651 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.188972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3c80-d30e-41ba-a1c3-023017f8f651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.190420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:24.215483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3f4b3c80-d30e-41ba-a1c3-023017f8f651 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:24.216703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3f4b3c80-d30e-41ba-a1c3-023017f8f651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:24.257312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d623cf09-63ef-44fe-b1c3-b1b4529c85f1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:24.294271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733900-36bb-4e70-9071-d9910e803246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:24.294963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733900-36bb-4e70-9071-d9910e803246 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.295252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733900-36bb-4e70-9071-d9910e803246 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.295561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733900-36bb-4e70-9071-d9910e803246 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.296241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733900-36bb-4e70-9071-d9910e803246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:24.297620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733900-36bb-4e70-9071-d9910e803246 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:24.304215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 385/774] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:33:24 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:24.383886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 390/775] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:24.418490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733900-36bb-4e70-9071-d9910e803246 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:24.546210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733900-36bb-4e70-9071-d9910e803246 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:24.548211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:24.576085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac86860-714d-45a5-a784-2b7207aca381 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.576955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac86860-714d-45a5-a784-2b7207aca381 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.576955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac86860-714d-45a5-a784-2b7207aca381 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.578015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 391/776] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:24.582233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:24.583126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388affe0-dfac-412d-a7d3-136bde544cca None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.583445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388affe0-dfac-412d-a7d3-136bde544cca None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.583732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388affe0-dfac-412d-a7d3-136bde544cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.583998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.584478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.584995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] PATH_INFO: `/v3/services/network` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.585835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:24.587452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:24.605432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:24.606466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:24.607325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:24.608811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-388affe0-dfac-412d-a7d3-136bde544cca None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:24.609997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-388affe0-dfac-412d-a7d3-136bde544cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:24.610659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 386/777] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:24 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:24.616065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:24.618440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:24.685452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 392/778] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:24.708211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388affe0-dfac-412d-a7d3-136bde544cca None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:24.740065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e10f19-4e6d-4e53-92e8-c5837b8dbfb1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:24.802782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:24.806213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.808081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.812956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.814041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] RBAC: Adding query filter params (name=network) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:24.814906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:24.817444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:24.838759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 387/779] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:24.846107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:24.851458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388affe0-dfac-412d-a7d3-136bde544cca None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:24.897141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b757b16-93d5-412f-8b08-224a5f5660d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:24.899102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b757b16-93d5-412f-8b08-224a5f5660d8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:24.900297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b757b16-93d5-412f-8b08-224a5f5660d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:24.900932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b757b16-93d5-412f-8b08-224a5f5660d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:24.909243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:24.929662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b757b16-93d5-412f-8b08-224a5f5660d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2b757b16-93d5-412f-8b08-224a5f5660d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:24.937923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:24.938937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:24.939797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:24.947974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 393/780] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:24.953874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b757b16-93d5-412f-8b08-224a5f5660d8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:24.961079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:25.039948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f15496-1891-4f19-af34-d457b59bad11 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:25.098345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:25.099410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:25.099911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:25.101012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:25.101966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] RBAC: Adding query filter params (type=network) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:25.102690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:25.104861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:25.117141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 388/781] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:25.119237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:25.143019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b757b16-93d5-412f-8b08-224a5f5660d8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:25.207986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:25.209006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:25.209488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:25.210002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:25.211029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:25.211555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:25.214592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:25.230884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 394/782] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:25.235962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:25.293646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825310c-21a0-44c9-a4c9-20a61de30d20 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:25.303154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:25.332365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:25.333389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:25.333909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:25.334401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:25.335671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] RBAC: Adding query filter params (interface=public, service_id=bb359451c45246dab862103c5135a5f5, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:25.336185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:25.337915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:25.346702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 389/783] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Jul 15 15:33:25 2022] GET /identity/v3/endpoints?service_id=bb359451c45246dab862103c5135a5f5&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:25.352843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:25.387669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:25.465779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93374c-f450-402b-aff7-96c9083c3a71 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:25.539070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:25.539908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:25.540195 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:25.540470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:25.557580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1ec58-84eb-4597-8ad1-bd1d03a7aa24 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:25.568673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:25.570302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:25.598428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:25.599370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:25.600230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:25.600230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:25.600230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:25.600230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:25.612045 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 395/784] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:25.617102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:25.630472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01eff-a574-4827-854f-38ecb8efb64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:25.631148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01eff-a574-4827-854f-38ecb8efb64e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:25.631467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01eff-a574-4827-854f-38ecb8efb64e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:25.631810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01eff-a574-4827-854f-38ecb8efb64e None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:25.632494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee01eff-a574-4827-854f-38ecb8efb64e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:25.633875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee01eff-a574-4827-854f-38ecb8efb64e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:25.646105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 390/785] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:33:25 2022] GET /identity/v3/services => generated 772 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:25.742556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac7969-7df2-43e0-8f40-9f02153996de None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:25.762059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:25.781742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa23fa1-593e-49fb-8d39-beea7f7e8c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:25.788008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa23fa1-593e-49fb-8d39-beea7f7e8c9c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:25.788008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa23fa1-593e-49fb-8d39-beea7f7e8c9c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:25.788008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa23fa1-593e-49fb-8d39-beea7f7e8c9c None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:25.788008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa23fa1-593e-49fb-8d39-beea7f7e8c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:25.788008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa23fa1-593e-49fb-8d39-beea7f7e8c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:25.788008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa23fa1-593e-49fb-8d39-beea7f7e8c9c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:25.799391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 396/786] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:25.805555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01eff-a574-4827-854f-38ecb8efb64e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:25.859669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:25.906755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85d6f-26fd-447c-9ad1-e92bda89f7c8 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:25.907459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85d6f-26fd-447c-9ad1-e92bda89f7c8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:25.907940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef85d6f-26fd-447c-9ad1-e92bda89f7c8 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:25.909810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 397/787] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:25.915983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb26f-4cff-476d-8432-5f202dbd1d1c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:25.916534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb26f-4cff-476d-8432-5f202dbd1d1c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:25.917008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb26f-4cff-476d-8432-5f202dbd1d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:25.948697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b6eeb26f-4cff-476d-8432-5f202dbd1d1c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:25.949941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b6eeb26f-4cff-476d-8432-5f202dbd1d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:25.962582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01eff-a574-4827-854f-38ecb8efb64e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.012916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.013904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.014195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.014477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.033727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:26.035227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:26.049126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 398/788] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:26.058598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:26.059506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:26.060422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 391/789] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:26.065436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eeb26f-4cff-476d-8432-5f202dbd1d1c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.076240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.180602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eeb26f-4cff-476d-8432-5f202dbd1d1c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.181806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:26.189247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:26.202836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cda21a-3b1a-48da-9c9b-afc8d66059c2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.229319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc251fe1-0964-4291-8360-99ed50026053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.230391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251fe1-0964-4291-8360-99ed50026053 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.231457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251fe1-0964-4291-8360-99ed50026053 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.231961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc251fe1-0964-4291-8360-99ed50026053 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.233289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc251fe1-0964-4291-8360-99ed50026053 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:26.234095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc251fe1-0964-4291-8360-99ed50026053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:26.236545 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc251fe1-0964-4291-8360-99ed50026053 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:26.247466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7588a-87e5-431c-8f6e-7f378dfacf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.248497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7588a-87e5-431c-8f6e-7f378dfacf26 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.248793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7588a-87e5-431c-8f6e-7f378dfacf26 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.249076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7588a-87e5-431c-8f6e-7f378dfacf26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.260756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 399/790] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:26.262198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1d6f-162f-4056-814c-b52d3870dc83 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.262509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1d6f-162f-4056-814c-b52d3870dc83 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.262811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a1d6f-162f-4056-814c-b52d3870dc83 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.268364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 400/791] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:26.268364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1d6f-162f-4056-814c-b52d3870dc83 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.270305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7588a-87e5-431c-8f6e-7f378dfacf26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b4d7588a-87e5-431c-8f6e-7f378dfacf26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:26.279716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:26.280665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:26.281525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:26.286147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 392/792] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:26.287450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97836-3392-4dd9-b8ad-e60d1ce45e6f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.287776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97836-3392-4dd9-b8ad-e60d1ce45e6f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.288600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c97836-3392-4dd9-b8ad-e60d1ce45e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.308817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-42c97836-3392-4dd9-b8ad-e60d1ce45e6f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:26.310179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-42c97836-3392-4dd9-b8ad-e60d1ce45e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:26.395569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 393/793] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:26.395569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c97836-3392-4dd9-b8ad-e60d1ce45e6f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.419770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a1d6f-162f-4056-814c-b52d3870dc83 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.468868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.470368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.471136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.472055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.481120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:26.487501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:26.490111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:26.513460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c97836-3392-4dd9-b8ad-e60d1ce45e6f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:26.514691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:26.515784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:26.516857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:26.516857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:26.516857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:26.516857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:26.519567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 401/794] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:26.520895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.545207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:26.557660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.558585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.567566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.567566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.567566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:26.567566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:26.567566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:26.575439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 394/795] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:26.576758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.646280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea2b8-677f-476e-90c9-e6c39def3c12 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.685789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070e097-bf46-4a03-b87a-5002bb40edfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.686931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070e097-bf46-4a03-b87a-5002bb40edfe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.687498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070e097-bf46-4a03-b87a-5002bb40edfe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.688013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070e097-bf46-4a03-b87a-5002bb40edfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.698986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070e097-bf46-4a03-b87a-5002bb40edfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:26.704848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaed966-46b5-4861-a656-3443e0016b6b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2070e097-bf46-4a03-b87a-5002bb40edfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:26.707115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:26.708040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:26.708856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:26.711574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 402/796] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:26.712715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070e097-bf46-4a03-b87a-5002bb40edfe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.748507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.749552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.750046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.750613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.751710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:26.752240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:26.754117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:26.756026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:26.768225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 395/797] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:26.774470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.816379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:26.845632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070e097-bf46-4a03-b87a-5002bb40edfe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.891075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d80b2d-7557-49f5-be8d-d2af4054825c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:26.896159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.896973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.897321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.897608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.911784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:26.914113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:26.930266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb03ac-c467-4308-a6ac-ce26ee6b8444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:26.930932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb03ac-c467-4308-a6ac-ce26ee6b8444 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.931223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb03ac-c467-4308-a6ac-ce26ee6b8444 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.931537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb03ac-c467-4308-a6ac-ce26ee6b8444 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.932282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb03ac-c467-4308-a6ac-ce26ee6b8444 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:26.932456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:26.933284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:26.934184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:26.934184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:26.934184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:26.934184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:26.934401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb03ac-c467-4308-a6ac-ce26ee6b8444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:26.934481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 403/798] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:26.935043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:26.938400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb03ac-c467-4308-a6ac-ce26ee6b8444 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:26.951134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 396/799] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:26.952584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5f05-f153-4d6e-a56f-cf791a8c4b6f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:26.952931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5f05-f153-4d6e-a56f-cf791a8c4b6f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:26.963493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb5f05-f153-4d6e-a56f-cf791a8c4b6f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:26.963493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 397/800] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:26.963493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb5f05-f153-4d6e-a56f-cf791a8c4b6f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.015687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:27.053285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0fdee-9a94-43cc-8dc8-4af3f1cc0a7d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.063796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb5f05-f153-4d6e-a56f-cf791a8c4b6f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.080335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:27.088203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.088995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.089277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.089559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.096765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.098227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.101617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f648b06-1556-4040-a48b-a8063665d1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.102454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f648b06-1556-4040-a48b-a8063665d1e3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.102739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f648b06-1556-4040-a48b-a8063665d1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.103015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f648b06-1556-4040-a48b-a8063665d1e3 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.103760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f648b06-1556-4040-a48b-a8063665d1e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:27.104071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f648b06-1556-4040-a48b-a8063665d1e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.105421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f648b06-1556-4040-a48b-a8063665d1e3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:27.112524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:27.113522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:27.114509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 404/801] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:27.114988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.122317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 398/802] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:27.123836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597879d-bc74-4af1-ad48-276f8166379e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.124262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597879d-bc74-4af1-ad48-276f8166379e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.124679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597879d-bc74-4af1-ad48-276f8166379e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.144223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7597879d-bc74-4af1-ad48-276f8166379e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:27.145398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7597879d-bc74-4af1-ad48-276f8166379e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:27.218652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 399/803] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:27.219342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597879d-bc74-4af1-ad48-276f8166379e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.240291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf3559-2f06-4ff8-bd3e-38e268062654 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.276338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:27.279048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.280052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.280505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.280928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.291611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.293288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:27.312496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:27.313370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:27.314271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:27.314271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:27.314271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:27.314271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:27.314271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 405/804] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:27.314550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.338897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597879d-bc74-4af1-ad48-276f8166379e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.358615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:27.376707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.377370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.377653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.377926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.378698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:27.379002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.382648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.394476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 400/805] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:27.400038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.424854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3a8e2-d861-4564-a14f-93b8524b836c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.463116 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.464266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.464704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.464974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.473049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.474640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:27.490598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:27.491771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 406/806] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:27.499175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.521414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19d971-0302-4ca2-a4bf-ee318ae3713c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.529583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:27.556114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.556925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.557221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.557503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.558158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.559561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:27.572600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:27.573500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:33:27.574309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:27.577400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 401/807] 10.222.0.98 () {56 vars in 1129 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:27.578148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.607948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:27.611006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167dc69-f22a-4406-82f7-9f8d3317cf18 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.646659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.647420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.647709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.647988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.648701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:27.649005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.650390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.666713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 407/808] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:27.666713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.693792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883cef7-1b0b-439b-8212-fb5501369a38 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.728973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1a976-a04b-44e4-957a-a7771a0281ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.729896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a976-a04b-44e4-957a-a7771a0281ca None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.730193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a976-a04b-44e4-957a-a7771a0281ca None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.730468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a976-a04b-44e4-957a-a7771a0281ca None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/8f69205a5fc042299a8913c924b45e45/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.732485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1a976-a04b-44e4-957a-a7771a0281ca None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=8f69205a5fc042299a8913c924b45e45, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.733928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1a976-a04b-44e4-957a-a7771a0281ca None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.768839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce18322-1fef-4c49-94a1-848196f4f09d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.802690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.803395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.804176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.804459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.805284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:27.805597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.807093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.807275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 402/809] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:27 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/8f69205a5fc042299a8913c924b45e45/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:33:27.811547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1a976-a04b-44e4-957a-a7771a0281ca None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.828211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 408/810] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:27.829020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.852173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:27.858826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:27.936360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1a976-a04b-44e4-957a-a7771a0281ca None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.948534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dd8a1-3233-4148-94b0-777c19191cc3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:27.978931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ee9fa-ab5f-4b3f-8398-f9a151f16734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.980070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee9fa-ab5f-4b3f-8398-f9a151f16734 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.980362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee9fa-ab5f-4b3f-8398-f9a151f16734 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.980642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee9fa-ab5f-4b3f-8398-f9a151f16734 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.981352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ee9fa-ab5f-4b3f-8398-f9a151f16734 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:27.981656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ee9fa-ab5f-4b3f-8398-f9a151f16734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:27.983093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ee9fa-ab5f-4b3f-8398-f9a151f16734 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:27.987908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 403/811] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:27.989184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957581d1-3958-49f3-820b-be69c2b175b9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.989471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957581d1-3958-49f3-820b-be69c2b175b9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.989749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957581d1-3958-49f3-820b-be69c2b175b9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:27.990787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:27.991504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 404/812] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:27.992162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957581d1-3958-49f3-820b-be69c2b175b9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:27.992726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:27.993268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:27.993831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.004287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.006001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:28.021336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:28.022847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:28.023808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:28.025650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 409/813] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:28.026792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.100224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:28.109347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:28.109689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957581d1-3958-49f3-820b-be69c2b175b9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:28.132930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482206-e270-4efe-8c89-257799c51b90 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:28.148100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11137d41-b7c8-42c0-9216-b88bacc47a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:28.149215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11137d41-b7c8-42c0-9216-b88bacc47a80 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.149503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11137d41-b7c8-42c0-9216-b88bacc47a80 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.149780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11137d41-b7c8-42c0-9216-b88bacc47a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.160356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11137d41-b7c8-42c0-9216-b88bacc47a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.161809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11137d41-b7c8-42c0-9216-b88bacc47a80 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.169659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:28.170339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.170657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.171025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.171768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:28.172086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.173475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-11137d41-b7c8-42c0-9216-b88bacc47a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:28.180928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:28.181775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:28.187801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:28.187801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:28.187801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:28.187801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:28.187801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 405/814] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:28.187801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68064853-1598-42ac-b271-f35afa44781e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.188127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 410/815] 10.222.0.98 () {56 vars in 1124 bytes} [Fri Jul 15 15:33:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:28.188198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68064853-1598-42ac-b271-f35afa44781e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.188198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68064853-1598-42ac-b271-f35afa44781e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.188345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.202303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-68064853-1598-42ac-b271-f35afa44781e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:28.203540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-68064853-1598-42ac-b271-f35afa44781e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:28.279338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 406/816] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:28.279338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68064853-1598-42ac-b271-f35afa44781e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.300494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f3e9e-23dc-48ec-b357-b8398d2c7e41 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:28.338052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:28.338770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.339059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.339365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.340252 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:28.340561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.342057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.357360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:28.366955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 411/817] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:33:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:28.366955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.368200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:28.390769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68064853-1598-42ac-b271-f35afa44781e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:28.433373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631f798-7e63-4ba2-9471-352c2f366754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:28.434037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631f798-7e63-4ba2-9471-352c2f366754 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.434324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631f798-7e63-4ba2-9471-352c2f366754 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.434600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631f798-7e63-4ba2-9471-352c2f366754 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.435370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631f798-7e63-4ba2-9471-352c2f366754 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.436758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631f798-7e63-4ba2-9471-352c2f366754 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.479052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54759b-7f66-405e-b96a-0b7e9ed970eb None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:28.512372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:28.513036 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.513321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.513597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.514308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:28.514612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.516017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.527795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 407/818] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:33:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:28.528268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 412/819] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:28.529185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631f798-7e63-4ba2-9471-352c2f366754 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.529307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.603199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:28.648577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0d8b3-47bb-4b2e-ab9a-946046a877ad None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:28.662862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631f798-7e63-4ba2-9471-352c2f366754 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:28.685810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c6514-fce4-4183-9d5c-f460caa30267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:28.686554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c6514-fce4-4183-9d5c-f460caa30267 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.686915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c6514-fce4-4183-9d5c-f460caa30267 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.687273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c6514-fce4-4183-9d5c-f460caa30267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.695183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:28.703868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:28.704663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.704957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.705219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] PATH_INFO: `/v3/services/network` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.705872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.707269 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:28.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:28.722868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:33:28.723917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:28.726808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 408/820] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:28 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:28.727500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.782581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c6514-fce4-4183-9d5c-f460caa30267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.784275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c6514-fce4-4183-9d5c-f460caa30267 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.836303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9960c-004e-4128-9993-6989d2fd97c4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:28.854814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:28.886340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:28.887149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:28.887478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:28.887760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:28.895534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:28.897097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:28.913196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:28.914314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:28.915436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 409/821] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:28.916148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.919383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 413/822] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:33:28.920259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c6514-fce4-4183-9d5c-f460caa30267 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:28.981163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:29.035816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb965-f52b-4498-aeac-e491dd7488eb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:29.067660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c6514-fce4-4183-9d5c-f460caa30267 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:29.077934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:29.079258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.079753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.080191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.081037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] RBAC: Adding query filter params (name=network) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:29.081489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:29.083024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:29.088749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 410/823] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:29.089701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:29.110586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:29.111948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.112334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.112712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.113857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:29.114262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:29.116659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:29.132859 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 414/824] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:29.133943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:29.159182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:29.248513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cf51d-3b1d-48f9-b304-0290e8f29fc1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:29.287434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:29.289219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.289660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.290097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.291250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] RBAC: Adding query filter params (type=network) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:29.292538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:29.294178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:29.300861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 415/825] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:29.302197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:29.325315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f053f5f-bae8-4f60-9378-6acba4969d75 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:29.344340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:29.364001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:29.364957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.365337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.365705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.381902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:29.381902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:29.405612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:29.406524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:29.407408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:29.407408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:29.407408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:29.407408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:29.411788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 411/826] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:29.412718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:29.526195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364656-e118-40e5-ab9e-f6e6f3fc8e27 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:29.543017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:29.560222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fc438-2003-4f84-94ca-611ef24d9e54 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:29.563835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d027ed-51a7-4d92-afa0-bcdb25a1e7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:29.564696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d027ed-51a7-4d92-afa0-bcdb25a1e7d3 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.565126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d027ed-51a7-4d92-afa0-bcdb25a1e7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.565545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d027ed-51a7-4d92-afa0-bcdb25a1e7d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.566435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d027ed-51a7-4d92-afa0-bcdb25a1e7d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:29.567963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d027ed-51a7-4d92-afa0-bcdb25a1e7d3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:29.596385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e98b2-1f7e-4c55-96a9-2b016a6e58c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:29.597407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e98b2-1f7e-4c55-96a9-2b016a6e58c6 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.597786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e98b2-1f7e-4c55-96a9-2b016a6e58c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.598155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e98b2-1f7e-4c55-96a9-2b016a6e58c6 None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/a1daa10e8ac5455cbf98f02971639237/roles/75570e166e6e4bd680698565f2d2cc0b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.613721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e98b2-1f7e-4c55-96a9-2b016a6e58c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=a1daa10e8ac5455cbf98f02971639237, role_id=75570e166e6e4bd680698565f2d2cc0b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:29.615167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e98b2-1f7e-4c55-96a9-2b016a6e58c6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:29.630168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 412/827] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:33:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:29.632549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d027ed-51a7-4d92-afa0-bcdb25a1e7d3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:29.665366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:29.722535 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 416/828] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:29 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/a1daa10e8ac5455cbf98f02971639237/roles/75570e166e6e4bd680698565f2d2cc0b => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:33:29.773191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d027ed-51a7-4d92-afa0-bcdb25a1e7d3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:29.815947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7384ef-3517-48b4-a633-5e20b3ea6c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:29.816597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7384ef-3517-48b4-a633-5e20b3ea6c1f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.818091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7384ef-3517-48b4-a633-5e20b3ea6c1f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.818091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7384ef-3517-48b4-a633-5e20b3ea6c1f None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.818091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7384ef-3517-48b4-a633-5e20b3ea6c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:29.818316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7384ef-3517-48b4-a633-5e20b3ea6c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:29.820630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7384ef-3517-48b4-a633-5e20b3ea6c1f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:29.834328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 413/829] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:29.840377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e98b2-1f7e-4c55-96a9-2b016a6e58c6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:29.959281 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beade95-1fe3-4b28-9284-4652d0f969ea None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.959617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beade95-1fe3-4b28-9284-4652d0f969ea None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.960474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beade95-1fe3-4b28-9284-4652d0f969ea None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.962118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 414/830] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:29.964685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e98b2-1f7e-4c55-96a9-2b016a6e58c6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:29.968116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c0567-2f6e-4b01-a8ff-037de0f0ada7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:29.968455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c0567-2f6e-4b01-a8ff-037de0f0ada7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:29.969618 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c0567-2f6e-4b01-a8ff-037de0f0ada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:29.994486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4c8c0567-2f6e-4b01-a8ff-037de0f0ada7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:29.995766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4c8c0567-2f6e-4b01-a8ff-037de0f0ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:30.008754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:30.009870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.010357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.010833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.013209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:30.019045 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:30.020812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:30.038201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:30.039123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:30.040103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:30.040103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:30.040103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:30.040103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:30.040103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:30.040103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:30.040103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:30.040103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:30.042660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 417/831] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:30.048044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:30.071102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 415/832] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:30.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c0567-2f6e-4b01-a8ff-037de0f0ada7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:30.112868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:30.184283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324569ee-7463-4aa0-8359-f5a5df828913 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:30.224462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c0567-2f6e-4b01-a8ff-037de0f0ada7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:30.233639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:30.233639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.233639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.233639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.234046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:30.234046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:30.235181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:30.253404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 418/833] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:30.259479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:30.270200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789226c-8c2a-4186-96d8-e5acd38a7579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:30.271010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789226c-8c2a-4186-96d8-e5acd38a7579 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.271414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789226c-8c2a-4186-96d8-e5acd38a7579 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.271694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789226c-8c2a-4186-96d8-e5acd38a7579 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.283925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789226c-8c2a-4186-96d8-e5acd38a7579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8789226c-8c2a-4186-96d8-e5acd38a7579 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:30.294770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:30.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:30.296581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 416/834] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:30.303109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789226c-8c2a-4186-96d8-e5acd38a7579 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:30.329324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:30.379523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:30.395632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa01d8-638c-45b0-93d5-fdd887ff0ef7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:30.413931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789226c-8c2a-4186-96d8-e5acd38a7579 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:30.445798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:30.446834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.447121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.447431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.448535 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=bf670978cc2e4c529093a191bc26f885) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:30.448833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:30.450283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:30.452498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:30.453402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.453890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.454392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.455370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:30.455882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:30.457483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:30.459441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 419/835] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:30 2022] GET /identity/v3/role_assignments?user.id=bf670978cc2e4c529093a191bc26f885&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:30.472125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 417/836] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:30.480661 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:30.600603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a393ab-0c11-4e1d-8963-47b290fbbfce None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:30.619835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:30.638815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4ed2e-e636-4ef6-901b-1921c733625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:30.649053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4ed2e-e636-4ef6-901b-1921c733625a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.649053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4ed2e-e636-4ef6-901b-1921c733625a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.649053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4ed2e-e636-4ef6-901b-1921c733625a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.651628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4ed2e-e636-4ef6-901b-1921c733625a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:30.653374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4ed2e-e636-4ef6-901b-1921c733625a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-3ac4ed2e-e636-4ef6-901b-1921c733625a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:30.670514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:30.671461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:30.672347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:30.672347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:30.672347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:30.672347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:30.672347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 420/837] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:30.679483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:30.725512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a432-c1b0-4f1c-b9de-48c8629fced5 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.725818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a432-c1b0-4f1c-b9de-48c8629fced5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.726084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82a432-c1b0-4f1c-b9de-48c8629fced5 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.727743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 421/838] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:30.732631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81573cd0-1251-4442-836f-89a21b22dab4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.733167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81573cd0-1251-4442-836f-89a21b22dab4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.733451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81573cd0-1251-4442-836f-89a21b22dab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.751696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-81573cd0-1251-4442-836f-89a21b22dab4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:30.752903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-81573cd0-1251-4442-836f-89a21b22dab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:30.802668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04955631-7c9b-4eba-bb09-91113c43471c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:30.823832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 422/839] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:30.836459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:30.841606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:30.842352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:30.842637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:30.842912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:30.849242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:30.849242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:30.849242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:30.852220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81573cd0-1251-4442-836f-89a21b22dab4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:30.863404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 418/840] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:30.869568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:30.985164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722af8dd-4dee-4e3f-813a-3e710656cc80 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:30.986009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:30.991816 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81573cd0-1251-4442-836f-89a21b22dab4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:31.029151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.029959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.030394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.030728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.038870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.039387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.040356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.040664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.041139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.041610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.042723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.044768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:31.058726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:31.059898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:31.065611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 419/841] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:31.066277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:31.067140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:31.068622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 423/842] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:31.072230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:31.072343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:31.129097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:31.177620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de169d4f-e5de-4f05-9cb6-b541eafa3661 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:31.212165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24397d-051e-4a2e-95d6-efa41e3adf95 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:31.220863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.221813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.222257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.222681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.223809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:31.224254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.226450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.249664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 420/843] 10.222.0.98 () {56 vars in 1135 bytes} [Fri Jul 15 15:33:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:31.252349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:31.260164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.260984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.261372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.261774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.262614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:31.263025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.264623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.269871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 424/844] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:31.274969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:31.306624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:31.376934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbf518-8b5f-4108-bce3-ce4bfb9ff599 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:31.397415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:31.404305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaab06d-3102-41f8-9da2-676a389b5f94 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:31.426103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce718d-4c52-42b4-bd4d-83c2cbf6591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.427399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce718d-4c52-42b4-bd4d-83c2cbf6591e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.427862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce718d-4c52-42b4-bd4d-83c2cbf6591e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.428338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce718d-4c52-42b4-bd4d-83c2cbf6591e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.448066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.449070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.449564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.450030 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.451005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:31.451547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.453186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.463723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 425/845] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:31 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:31.507505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce718d-4c52-42b4-bd4d-83c2cbf6591e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.508984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce718d-4c52-42b4-bd4d-83c2cbf6591e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.624931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 421/846] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:33:31.626338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:31.729643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:31.739578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560319a0-3265-4768-8b9c-23b2e96a1214 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:31.776433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3acdd3-ceb8-43ea-a29b-fc547ff1692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.777864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3acdd3-ceb8-43ea-a29b-fc547ff1692e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.778369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3acdd3-ceb8-43ea-a29b-fc547ff1692e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.778826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3acdd3-ceb8-43ea-a29b-fc547ff1692e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.789946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3acdd3-ceb8-43ea-a29b-fc547ff1692e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.791774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3acdd3-ceb8-43ea-a29b-fc547ff1692e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ce3acdd3-ceb8-43ea-a29b-fc547ff1692e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:31.810468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:31.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:31.812458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:31.812458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:31.812458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:31.812458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:31.818694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 426/847] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:31.820255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce718d-4c52-42b4-bd4d-83c2cbf6591e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:31.832539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:31.923444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce718d-4c52-42b4-bd4d-83c2cbf6591e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:31.945740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b70353-5105-4cfc-952f-6b130b4e845d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.952318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b70353-5105-4cfc-952f-6b130b4e845d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.952318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b70353-5105-4cfc-952f-6b130b4e845d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.952318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 427/848] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:31.954149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f250a-3bf8-4cb5-9dfd-97136012d3ce None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.954651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f250a-3bf8-4cb5-9dfd-97136012d3ce None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.955124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f250a-3bf8-4cb5-9dfd-97136012d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.959967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.960634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.960917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.961192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.962031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:31.962340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.963896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.974531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9f5f250a-3bf8-4cb5-9dfd-97136012d3ce None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:31.976564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9f5f250a-3bf8-4cb5-9dfd-97136012d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:31.977053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 422/849] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:31.983946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.059392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 428/850] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:32.060530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:32.077325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f250a-3bf8-4cb5-9dfd-97136012d3ce None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.093521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a76606-f9f6-41b9-8d71-5d91668c9cea None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.131325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.132749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.133037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.133366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.141592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:32.143216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:32.157586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:32.160080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:32.160956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:32.161805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:32.161805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:32.161805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:32.161805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:32.161805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:32.161805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:32.161805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:32.161805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:32.164879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 423/851] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:32.175068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.188744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f250a-3bf8-4cb5-9dfd-97136012d3ce None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.224738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c2dcb-500b-4c75-81ab-4e4c12fb9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.225706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2dcb-500b-4c75-81ab-4e4c12fb9164 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.225979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2dcb-500b-4c75-81ab-4e4c12fb9164 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.226371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2dcb-500b-4c75-81ab-4e4c12fb9164 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.237112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c2dcb-500b-4c75-81ab-4e4c12fb9164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-539c2dcb-500b-4c75-81ab-4e4c12fb9164 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:32.245299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:32.246298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:33:32.247120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:32.251413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 429/852] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:32.252956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c2dcb-500b-4c75-81ab-4e4c12fb9164 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.291561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48229fb4-2a06-4d97-ae5f-aa4e98879db2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.323686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:32.330161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.331089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.331704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.332209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.333324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:32.333936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:32.336188 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:32.352920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 424/853] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:32.355648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.358359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c2dcb-500b-4c75-81ab-4e4c12fb9164 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.397142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.398338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.398621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.398896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.399808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:32.400112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:32.401554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:32.404006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:32.415309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 430/854] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:32.420294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.468879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4aec5a-1c51-4f5f-a153-a67817104dc9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.506212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.507265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.507907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.508387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.509473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=8f69205a5fc042299a8913c924b45e45) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:32.509996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:32.511776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:32.527676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 425/855] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/role_assignments?user.id=8f69205a5fc042299a8913c924b45e45&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:32.528286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd8bd8-cabf-44a3-9102-c76aa5a96113 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.568200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bd3bc-7936-4666-8b6a-74a46265b622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.569061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd3bc-7936-4666-8b6a-74a46265b622 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.569921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd3bc-7936-4666-8b6a-74a46265b622 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.570200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bd3bc-7936-4666-8b6a-74a46265b622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.582353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bd3bc-7936-4666-8b6a-74a46265b622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:32.583994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bd3bc-7936-4666-8b6a-74a46265b622 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-8d3bd3bc-7936-4666-8b6a-74a46265b622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:32.604735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:32.605691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:32.606561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:32.606561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:32.606561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:32.606561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:32.610805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 431/856] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:32.612440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.641776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc5f0a-7cb1-46d8-95cb-9c479cfa3ccd None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.642146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc5f0a-7cb1-46d8-95cb-9c479cfa3ccd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.642335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc5f0a-7cb1-46d8-95cb-9c479cfa3ccd None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.644135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 432/857] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:32.650134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08472ece-a3cc-4aa3-957f-cc5a7992bc15 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.650478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08472ece-a3cc-4aa3-957f-cc5a7992bc15 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.650686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08472ece-a3cc-4aa3-957f-cc5a7992bc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.675329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-08472ece-a3cc-4aa3-957f-cc5a7992bc15 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:32.679319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-08472ece-a3cc-4aa3-957f-cc5a7992bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:32.691734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:32.735983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:32.745150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f295ec-e4be-4829-9db3-aec5fc6be6c3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.763230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 433/858] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:32.787145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08472ece-a3cc-4aa3-957f-cc5a7992bc15 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.787794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.788978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.789265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.789545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.790266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:32.790662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:32.792528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:32.806644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 426/859] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:32.812578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:32.916030 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08472ece-a3cc-4aa3-957f-cc5a7992bc15 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.927806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e89843-5e57-4760-86ec-329fcc78e1ed None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:32.951744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:32.966562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.967916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.968542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.969149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.969969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:32.970118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:32.971037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:32.971552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:32.971930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:32.972199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:32.981107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:32.984304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:32.988971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:32.989968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:32.991266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 434/860] 10.222.0.98 () {56 vars in 1132 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:32.999090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:33.008941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:33.009919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:33.010862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:33.018841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 427/861] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:33.020728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:33.066923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:33.136892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf382481-c5d3-4a11-b3e8-f86c3b421c40 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:33.143746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837abf-b96e-4baa-9d09-934c73ed593c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:33.180287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:33.181443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.181905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:33.182177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.182655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.182913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.182999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.183378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.184118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:33.184502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:33.184796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:33.184898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:33.186364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:33.186641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:33.193651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 435/862] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:33.200276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:33.213632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 428/863] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Jul 15 15:33:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:33.215283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:33.248190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:33.315471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcd6bc-0354-44c3-99de-4cc91039aadc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:33.323994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:33.330925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea0fba-c4e4-47e6-995f-750884ae66bd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:33.359711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:33.360474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.360862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.361234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.362059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:33.362465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:33.364053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:33.371470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acdcc2-aaee-414a-a038-be6f57d54bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:33.372326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acdcc2-aaee-414a-a038-be6f57d54bcd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.372773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acdcc2-aaee-414a-a038-be6f57d54bcd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.373190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acdcc2-aaee-414a-a038-be6f57d54bcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.375825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 436/864] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:33.457673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acdcc2-aaee-414a-a038-be6f57d54bcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:33.459426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acdcc2-aaee-414a-a038-be6f57d54bcd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:33.494231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:33.574783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 429/865] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:33:33.576378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:33.594680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1256cb6-f1df-4aae-aed0-3036099c615a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.595196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1256cb6-f1df-4aae-aed0-3036099c615a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.595647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1256cb6-f1df-4aae-aed0-3036099c615a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.597172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 430/866] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:33.604969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec75eb-0cd1-43de-805f-0412e21b8026 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.605473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec75eb-0cd1-43de-805f-0412e21b8026 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.605757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec75eb-0cd1-43de-805f-0412e21b8026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.623541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-38ec75eb-0cd1-43de-805f-0412e21b8026 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:33.624749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-38ec75eb-0cd1-43de-805f-0412e21b8026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:33.697432 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1e86a-b123-448b-ba77-c8db4dee729b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:33.714328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 431/867] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:33.737206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec75eb-0cd1-43de-805f-0412e21b8026 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:33.743011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:33.744118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.744697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.745191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.759227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:33.761236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:33.782347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:33.783332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:33.784242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:33.784242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:33.784242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:33.784242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:33.791059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 437/868] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:33.792686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:33.809281 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:33.822304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:33.859715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec75eb-0cd1-43de-805f-0412e21b8026 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:33.906304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f265b6c-986d-4c7c-8976-574a88f684d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:33.906980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f265b6c-986d-4c7c-8976-574a88f684d3 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.907267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f265b6c-986d-4c7c-8976-574a88f684d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.907572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f265b6c-986d-4c7c-8976-574a88f684d3 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.908305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f265b6c-986d-4c7c-8976-574a88f684d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:33.909061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e2f0b-d9ef-4f96-9d2c-cc87d9b771ee None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:33.909669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f265b6c-986d-4c7c-8976-574a88f684d3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:33.946040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:33.946710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:33.946998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:33.947280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:33.948025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:33.948511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:33.949777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:33.962231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 438/869] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:33.967755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:34.000098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 432/870] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:33:33 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:34.067653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:34.089205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4990d21-ab77-45a2-8c58-f84a4064d755 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:34.134817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:34.135631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:34.135976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:34.136246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:34.144146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:34.145729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:34.161244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:34.162237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:34.163709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 439/871] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:34.170899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f265b6c-986d-4c7c-8976-574a88f684d3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:34.280644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f265b6c-986d-4c7c-8976-574a88f684d3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:34.305155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:34.317576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa0a4f-7e33-48de-96b0-da6c720c68e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:34.318623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0a4f-7e33-48de-96b0-da6c720c68e2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:34.318623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0a4f-7e33-48de-96b0-da6c720c68e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:34.318854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa0a4f-7e33-48de-96b0-da6c720c68e2 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:34.320189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa0a4f-7e33-48de-96b0-da6c720c68e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:34.320507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa0a4f-7e33-48de-96b0-da6c720c68e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:34.322021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa0a4f-7e33-48de-96b0-da6c720c68e2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:34.339146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 433/872] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:34.340522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:34.460658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5e22a-3222-490a-a936-5742e0f8f39a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:34.496307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf245d-780a-42fc-bbbe-e31a7feb8522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:34.497372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf245d-780a-42fc-bbbe-e31a7feb8522 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:34.497863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf245d-780a-42fc-bbbe-e31a7feb8522 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:34.498351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf245d-780a-42fc-bbbe-e31a7feb8522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:34.499405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf245d-780a-42fc-bbbe-e31a7feb8522 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=a1daa10e8ac5455cbf98f02971639237) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:34.499944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf245d-780a-42fc-bbbe-e31a7feb8522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:34.501601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf245d-780a-42fc-bbbe-e31a7feb8522 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:34.515793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 440/873] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:34 2022] GET /identity/v3/role_assignments?user.id=a1daa10e8ac5455cbf98f02971639237&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:34.693637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b24e-9bf1-40d0-8373-1f7b507bd49a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:34.696930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b24e-9bf1-40d0-8373-1f7b507bd49a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:34.696930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4b24e-9bf1-40d0-8373-1f7b507bd49a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:34.696930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 434/874] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:34.701081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8897ef2-8897-423a-bf20-1d8b7f0339f1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:34.701612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8897ef2-8897-423a-bf20-1d8b7f0339f1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:34.702095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8897ef2-8897-423a-bf20-1d8b7f0339f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:34.708101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:34.720110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d8897ef2-8897-423a-bf20-1d8b7f0339f1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:34.721315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d8897ef2-8897-423a-bf20-1d8b7f0339f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:34.794825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 441/875] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:34.816267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4b24e-9bf1-40d0-8373-1f7b507bd49a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:34.924822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4b24e-9bf1-40d0-8373-1f7b507bd49a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:34.963313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:34.963313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:34.963313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:34.963313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:34.964528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:34.966099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:34.979533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:34.980526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:34.981334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:34.983805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 435/876] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:34.991548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8897ef2-8897-423a-bf20-1d8b7f0339f1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:35.103740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8897ef2-8897-423a-bf20-1d8b7f0339f1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:35.135116 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:35.140227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368124ec-a265-494f-8d9f-da01427c809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:35.140919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368124ec-a265-494f-8d9f-da01427c809b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:35.141216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368124ec-a265-494f-8d9f-da01427c809b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:35.141493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368124ec-a265-494f-8d9f-da01427c809b None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:35.142203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368124ec-a265-494f-8d9f-da01427c809b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:35.142515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368124ec-a265-494f-8d9f-da01427c809b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:35.143928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368124ec-a265-494f-8d9f-da01427c809b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:35.149075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 442/877] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:35.155062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:35.206099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:35.262276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5e27-4920-41e6-acb1-e52a8c1d5d85 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:35.299547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187eb88-8e69-40cd-9151-ea6f591b952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:35.300345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187eb88-8e69-40cd-9151-ea6f591b952b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:35.300636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187eb88-8e69-40cd-9151-ea6f591b952b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:35.300915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187eb88-8e69-40cd-9151-ea6f591b952b None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:35.301626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187eb88-8e69-40cd-9151-ea6f591b952b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:35.301933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187eb88-8e69-40cd-9151-ea6f591b952b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:35.303322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187eb88-8e69-40cd-9151-ea6f591b952b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:35.308593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 436/878] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:35.455020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12bd7d-9682-4f57-94c4-b7a887d0ec6e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:35.455437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12bd7d-9682-4f57-94c4-b7a887d0ec6e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:35.455781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12bd7d-9682-4f57-94c4-b7a887d0ec6e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:35.457477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 443/879] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:35.462727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f72901-cbd5-4b2c-81a2-c709f57d5534 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:35.463086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f72901-cbd5-4b2c-81a2-c709f57d5534 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:35.463489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f72901-cbd5-4b2c-81a2-c709f57d5534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:35.481779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b8f72901-cbd5-4b2c-81a2-c709f57d5534 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:35.483014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b8f72901-cbd5-4b2c-81a2-c709f57d5534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:35.551247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 437/880] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:35 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:35.573774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12bd7d-9682-4f57-94c4-b7a887d0ec6e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:35.677454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12bd7d-9682-4f57-94c4-b7a887d0ec6e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:35.712087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdb932-765b-498c-bb43-d12c06750947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:35.712758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdb932-765b-498c-bb43-d12c06750947 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:35.713042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdb932-765b-498c-bb43-d12c06750947 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:35.713319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdb932-765b-498c-bb43-d12c06750947 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:35.714046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdb932-765b-498c-bb43-d12c06750947 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:35.715459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdb932-765b-498c-bb43-d12c06750947 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:35.823560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 444/881] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:33:35 2022] POST /identity/v3/services => generated 269 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:36.209633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633adf10-907c-4152-89e7-ff6f5a7632e7 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:36.210173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633adf10-907c-4152-89e7-ff6f5a7632e7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:36.210448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633adf10-907c-4152-89e7-ff6f5a7632e7 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:36.211852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 438/882] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:36.217181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61998acd-223f-4cfc-b4aa-94b13cc5afeb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:36.217590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61998acd-223f-4cfc-b4aa-94b13cc5afeb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:36.217786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61998acd-223f-4cfc-b4aa-94b13cc5afeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:36.235151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-61998acd-223f-4cfc-b4aa-94b13cc5afeb None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:36.236404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-61998acd-223f-4cfc-b4aa-94b13cc5afeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:36.288677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:36.311908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 445/883] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:36.334026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633adf10-907c-4152-89e7-ff6f5a7632e7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:36.388967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:36.456894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633adf10-907c-4152-89e7-ff6f5a7632e7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:36.492357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:36.493080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:36.493364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:36.493642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:36.494289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:36.495693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:36.508183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:36.509202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:36.510013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 439/884] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:36.515801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61998acd-223f-4cfc-b4aa-94b13cc5afeb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:36.615425 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61998acd-223f-4cfc-b4aa-94b13cc5afeb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:36.659170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a1128-96cb-4b43-b5cf-4578957a1b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:36.659908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a1128-96cb-4b43-b5cf-4578957a1b41 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:36.660195 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a1128-96cb-4b43-b5cf-4578957a1b41 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:36.660474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a1128-96cb-4b43-b5cf-4578957a1b41 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:36.661184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a1128-96cb-4b43-b5cf-4578957a1b41 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:36.661490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a1128-96cb-4b43-b5cf-4578957a1b41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:36.662868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a1128-96cb-4b43-b5cf-4578957a1b41 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:36.669651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 446/885] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:36.675388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:36.783923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e50421-580b-4aa4-b489-f390d90bb2eb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:36.786211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:36.818238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee216f-44ee-4e22-ba7d-8d12d349f794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:36.818905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee216f-44ee-4e22-ba7d-8d12d349f794 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:36.819190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee216f-44ee-4e22-ba7d-8d12d349f794 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:36.819548 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee216f-44ee-4e22-ba7d-8d12d349f794 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:36.820254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee216f-44ee-4e22-ba7d-8d12d349f794 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:36.820562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee216f-44ee-4e22-ba7d-8d12d349f794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:36.822050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee216f-44ee-4e22-ba7d-8d12d349f794 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:36.822534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8fc6c-69ab-40d5-a006-4a37e142f3b2 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:36.822830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8fc6c-69ab-40d5-a006-4a37e142f3b2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:36.823108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8fc6c-69ab-40d5-a006-4a37e142f3b2 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:36.824654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 447/886] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:36.827470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 440/887] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:36.833625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8fc6c-69ab-40d5-a006-4a37e142f3b2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:36.834358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c2add-deb2-4ea4-bd86-845e5462271e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:36.834655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c2add-deb2-4ea4-bd86-845e5462271e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:36.835481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c2add-deb2-4ea4-bd86-845e5462271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:36.855929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2f2c2add-deb2-4ea4-bd86-845e5462271e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:36.857178 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2f2c2add-deb2-4ea4-bd86-845e5462271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:36.887123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:36.939571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 441/888] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:36.943797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8fc6c-69ab-40d5-a006-4a37e142f3b2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:36.957648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c2add-deb2-4ea4-bd86-845e5462271e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:36.979841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:36.980788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:36.981227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:36.981659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:36.982695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] RBAC: Adding query filter params (interface=public, service_id=219d6fb5d9f442cfbb055eb6a0f80265, region_id=RegionOne) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:36.983157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:36.984813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:36.991459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 448/889] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Jul 15 15:33:36 2022] GET /identity/v3/endpoints?service_id=219d6fb5d9f442cfbb055eb6a0f80265&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:36.996742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:37.079004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:37.084801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c2add-deb2-4ea4-bd86-845e5462271e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:37.133162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04e8a-47c7-4bd0-bacd-61c8b8355ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:37.135029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04e8a-47c7-4bd0-bacd-61c8b8355ee0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.135495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04e8a-47c7-4bd0-bacd-61c8b8355ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.135924 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d04e8a-47c7-4bd0-bacd-61c8b8355ee0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.136870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c07a7-dd61-4bc4-9434-95b958f26380 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:37.148834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:37.148993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d04e8a-47c7-4bd0-bacd-61c8b8355ee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e2d04e8a-47c7-4bd0-bacd-61c8b8355ee0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:37.159637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:37.163639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:37.164452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 442/890] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Jul 15 15:33:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:37.167979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04e8a-47c7-4bd0-bacd-61c8b8355ee0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:37.180888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c9af7-7cfd-42f3-8fc5-5c82ebc01813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:37.181862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c9af7-7cfd-42f3-8fc5-5c82ebc01813 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.182401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c9af7-7cfd-42f3-8fc5-5c82ebc01813 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.183034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c9af7-7cfd-42f3-8fc5-5c82ebc01813 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.184160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c9af7-7cfd-42f3-8fc5-5c82ebc01813 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:37.185796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c9af7-7cfd-42f3-8fc5-5c82ebc01813 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:37.193579 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 449/891] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:33:36 2022] GET /identity/v3/services => generated 1268 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:33:37.296706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d04e8a-47c7-4bd0-bacd-61c8b8355ee0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:37.337158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25edeb-4c67-449e-a8c2-00d66e2ecb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:37.338226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25edeb-4c67-449e-a8c2-00d66e2ecb08 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.338708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25edeb-4c67-449e-a8c2-00d66e2ecb08 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.339181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25edeb-4c67-449e-a8c2-00d66e2ecb08 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.340173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25edeb-4c67-449e-a8c2-00d66e2ecb08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:37.340738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25edeb-4c67-449e-a8c2-00d66e2ecb08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:37.342398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25edeb-4c67-449e-a8c2-00d66e2ecb08 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:37.352834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 443/892] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Jul 15 15:33:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:37.358710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c9af7-7cfd-42f3-8fc5-5c82ebc01813 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:37.384123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925dd4c-4cc9-498e-8f83-4db2d2c77a12 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.384737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925dd4c-4cc9-498e-8f83-4db2d2c77a12 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.385270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925dd4c-4cc9-498e-8f83-4db2d2c77a12 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.386639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 444/893] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:37.391944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3a986-5d2e-4866-bc01-07b45fb7c760 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.392849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3a986-5d2e-4866-bc01-07b45fb7c760 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.392849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3a986-5d2e-4866-bc01-07b45fb7c760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.413828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d6e3a986-5d2e-4866-bc01-07b45fb7c760 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:37.415018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d6e3a986-5d2e-4866-bc01-07b45fb7c760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:37.429775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:37.486703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c9af7-7cfd-42f3-8fc5-5c82ebc01813 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:37.494824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 445/894] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:37.519093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3a986-5d2e-4866-bc01-07b45fb7c760 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:37.532777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:37.533869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.534369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.534841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.546603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:37.548434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:37.567152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:37.568146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:37.569001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:37.569001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:37.569001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:37.569001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:37.575364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 450/895] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:37.576981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:37.606665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:37.638783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3a986-5d2e-4866-bc01-07b45fb7c760 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:37.683477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882fbfd-8983-40a7-b299-9e2bd93e4392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:37.684169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882fbfd-8983-40a7-b299-9e2bd93e4392 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.684459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882fbfd-8983-40a7-b299-9e2bd93e4392 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.684741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882fbfd-8983-40a7-b299-9e2bd93e4392 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.685764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882fbfd-8983-40a7-b299-9e2bd93e4392 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:37.688493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882fbfd-8983-40a7-b299-9e2bd93e4392 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:37.701295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4d70e-93c1-46d9-9998-7086a4202ca7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:37.745987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:37.746658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.746940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.747214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.747967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:37.748381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:37.749770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:37.763790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 446/896] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:33:37 2022] POST /identity/v3/services => generated 241 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:37.765448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 451/897] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:37.771514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882fbfd-8983-40a7-b299-9e2bd93e4392 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:37.791536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:37.899106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882fbfd-8983-40a7-b299-9e2bd93e4392 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:37.936295 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b20291-69af-4088-8c4d-ca6d85b23149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:37.937148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b20291-69af-4088-8c4d-ca6d85b23149 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.937383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b20291-69af-4088-8c4d-ca6d85b23149 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.937657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b20291-69af-4088-8c4d-ca6d85b23149 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.946515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b20291-69af-4088-8c4d-ca6d85b23149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:37.948000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b20291-69af-4088-8c4d-ca6d85b23149 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-55b20291-69af-4088-8c4d-ca6d85b23149 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:37.963217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:37.964240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:37.965172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 447/898] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:33:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:37.970648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:37.972841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428532-d4ab-430a-ade8-bbe72495c2be None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.973125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428532-d4ab-430a-ade8-bbe72495c2be None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.973414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428532-d4ab-430a-ade8-bbe72495c2be None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:37.976260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 448/899] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:37.981515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0d5df-96ed-4d04-bf23-bb9f1888042c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:37.982313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0d5df-96ed-4d04-bf23-bb9f1888042c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:37.982600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0d5df-96ed-4d04-bf23-bb9f1888042c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.004591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8af0d5df-96ed-4d04-bf23-bb9f1888042c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:38.005916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8af0d5df-96ed-4d04-bf23-bb9f1888042c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:38.089796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:38.092493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:38.111455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 449/900] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:38.111651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce0dc9-2782-43ac-ad14-06494206005e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:38.129667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0d5df-96ed-4d04-bf23-bb9f1888042c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.155548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:38.156549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:38.157429 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:38.157886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.159166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:38.159942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:38.162174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:38.189246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 452/901] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Jul 15 15:33:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:38.190945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.248530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0d5df-96ed-4d04-bf23-bb9f1888042c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:38.287220 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:38.288471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:38.288951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:38.289438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.290373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:38.292064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:38.302919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372c951-f6a2-4137-9e1f-c6d1ca457002 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:38.306881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:38.308140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:38.308949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:38.316414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 450/902] 10.222.0.98 () {56 vars in 1132 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:38.317511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.347317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aef5b4-c820-4902-92c4-822f829554d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:38.347317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aef5b4-c820-4902-92c4-822f829554d5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:38.347317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aef5b4-c820-4902-92c4-822f829554d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:38.347317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aef5b4-c820-4902-92c4-822f829554d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.362165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:38.387071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:38.440105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aef5b4-c820-4902-92c4-822f829554d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:38.442181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aef5b4-c820-4902-92c4-822f829554d5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:38.447784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0a8db-7483-4aac-8a18-7105a3e2fdda None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:38.485841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:38.486788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:38.487192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:38.487592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.488694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:38.488886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:38.490341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:38.496505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 451/903] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:38.501745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.576473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 453/904] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:33:38.584402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aef5b4-c820-4902-92c4-822f829554d5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.614947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d8ca2-73ae-408c-84b2-aa438d0f56da None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:38.631648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:38.639201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:38.650018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:38.652862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:38.652862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:38.652862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.652862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:38.652862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:38.653944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:38.659389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 452/905] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:38.665395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.694810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aef5b4-c820-4902-92c4-822f829554d5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:38.732392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:38.733261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:38.733551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:38.733872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.745609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:38.747245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:38.766207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:38.767137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:38.768054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:38.768054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:38.768054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:38.768054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:38.772240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 454/906] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:38.773346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.792659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30da1b-469e-47af-8031-975849b01b7d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:38.829858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:38.830913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:38.831452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:38.831972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.832958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] RBAC: Adding query filter params (interface=public, service_id=d72eafd6c2f64cd79a5f3aecc18f465d, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:38.833477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:38.836321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:38.846835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 453/907] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/endpoints?service_id=d72eafd6c2f64cd79a5f3aecc18f465d&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:38.848123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.884033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe968cb5-e4a4-4919-b990-3f829d812511 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:38.899636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:38.906994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:38.922809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:38.923249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:38.923565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:38.923842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:38.924567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:38.924873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:38.926295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:38.938921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 455/908] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:38.944706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:38.963048 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc486870-b724-4842-ae0e-72f02e49ac73 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:39.008176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef7887-c672-432d-a0c6-37982395293d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:39.009627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef7887-c672-432d-a0c6-37982395293d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.010412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef7887-c672-432d-a0c6-37982395293d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.011089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef7887-c672-432d-a0c6-37982395293d None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.012022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef7887-c672-432d-a0c6-37982395293d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:39.013972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef7887-c672-432d-a0c6-37982395293d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:39.026839 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 454/909] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:33:39.065535 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704fac8-1288-4943-b92c-649e2bfdb316 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:39.107842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:39.108305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.108590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.108867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.117413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:39.119023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:39.135637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:39.136587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:39.137445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 456/910] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:39.143039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef7887-c672-432d-a0c6-37982395293d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:39.261384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef7887-c672-432d-a0c6-37982395293d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:39.279843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:39.298463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9e9db-60ec-4d5f-8e75-b732afa8553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:39.299461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e9db-60ec-4d5f-8e75-b732afa8553b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.299931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e9db-60ec-4d5f-8e75-b732afa8553b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.300390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e9db-60ec-4d5f-8e75-b732afa8553b None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.302602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9e9db-60ec-4d5f-8e75-b732afa8553b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:39.303115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9e9db-60ec-4d5f-8e75-b732afa8553b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:39.304819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9e9db-60ec-4d5f-8e75-b732afa8553b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:39.318411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 455/911] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:39.325693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:39.345680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:39.412795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d2a33-9759-4e61-9d44-66c6a65ef993 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.413118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d2a33-9759-4e61-9d44-66c6a65ef993 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.413399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d2a33-9759-4e61-9d44-66c6a65ef993 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.416162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 456/912] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:39.420574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbea74-6283-4f84-9547-406420439e93 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.420866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbea74-6283-4f84-9547-406420439e93 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.421152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbea74-6283-4f84-9547-406420439e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.435223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f9ef2-b590-4eb8-9ab9-ad03778b0554 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:39.443610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-81bbea74-6283-4f84-9547-406420439e93 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:39.445391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-81bbea74-6283-4f84-9547-406420439e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:39.483102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:39.484051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.484452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.484739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.485573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] RBAC: Adding query filter params (role.id=973b62e0cb57414ca5df0f3a32e38887, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=a1daa10e8ac5455cbf98f02971639237) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:39.485887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:39.487423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:39.499419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 457/913] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:39 2022] GET /identity/v3/role_assignments?user.id=a1daa10e8ac5455cbf98f02971639237&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=973b62e0cb57414ca5df0f3a32e38887 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:39.543610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 457/914] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:39.568274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:39.673223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:39.693583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e211d-6299-4843-a876-37cf19201410 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:39.730286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:39.731096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.731405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.731680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.732325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:39.733764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:39.746378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:39.747449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:39.748435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 458/915] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:39.755834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbea74-6283-4f84-9547-406420439e93 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:39.865241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbea74-6283-4f84-9547-406420439e93 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:39.873266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:39.906423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84a0d6-bc9c-4502-bdbf-5125180756aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:39.907086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84a0d6-bc9c-4502-bdbf-5125180756aa None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.907397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84a0d6-bc9c-4502-bdbf-5125180756aa None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.907676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84a0d6-bc9c-4502-bdbf-5125180756aa None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.908375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84a0d6-bc9c-4502-bdbf-5125180756aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:39.908678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84a0d6-bc9c-4502-bdbf-5125180756aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:39.910051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84a0d6-bc9c-4502-bdbf-5125180756aa None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:39.915530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 458/916] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:39.924151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:39.949887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74d83-0e86-443a-b489-2347a8a7b78c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.950217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74d83-0e86-443a-b489-2347a8a7b78c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.950493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74d83-0e86-443a-b489-2347a8a7b78c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.952008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 459/917] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:39.959460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a8fe3-829a-44e6-a2c5-97fb97460e8d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:39.960234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a8fe3-829a-44e6-a2c5-97fb97460e8d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:39.969021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a8fe3-829a-44e6-a2c5-97fb97460e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:39.981936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4d1a8fe3-829a-44e6-a2c5-97fb97460e8d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:39.983401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4d1a8fe3-829a-44e6-a2c5-97fb97460e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:40.051861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e76f9-de8e-4446-9d29-b57598e81a0c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:40.061676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 460/918] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:40.088145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:40.088813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:40.089098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:40.089420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:40.090369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:40.090668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:40.091320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a8fe3-829a-44e6-a2c5-97fb97460e8d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:40.092089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:40.097544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 459/919] 10.222.0.98 () {56 vars in 1114 bytes} [Fri Jul 15 15:33:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:40.103769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:40.119467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:40.210378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a8fe3-829a-44e6-a2c5-97fb97460e8d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:40.212436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:40.232527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77949e-5403-4d07-8ade-526512c928c3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:40.257901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:40.258712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:40.258997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:40.259276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:40.259952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:40.262042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:40.274956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8f75-135f-40d5-97cb-dddcbac71e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:40.275973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8f75-135f-40d5-97cb-dddcbac71e9b None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:40.276464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8f75-135f-40d5-97cb-dddcbac71e9b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:40.276734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:40.277922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:40.284122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 461/920] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:40.284917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad8f75-135f-40d5-97cb-dddcbac71e9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:40.284917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad8f75-135f-40d5-97cb-dddcbac71e9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:40.284917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad8f75-135f-40d5-97cb-dddcbac71e9b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:40.285106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:40.375543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 460/921] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:33:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:33:40.390685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd602f-a64a-4c33-8a69-64f0b1ba2639 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:40.440670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:40.441806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:40.442312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:40.442797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:40.443740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:40.444252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:40.446433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:40.452697 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 462/922] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:40.458988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8f75-135f-40d5-97cb-dddcbac71e9b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:40.538829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:40.580874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad8f75-135f-40d5-97cb-dddcbac71e9b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:40.620118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:40.621185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:40.621674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:40.622149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:40.623064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:40.623609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:40.625193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:40.630883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 461/923] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:40.637086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:40.664504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:40.745431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57ae7-067f-4be5-ab97-be88f2599d59 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:40.781437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e7c60-e186-42c9-9f66-8a44b6204622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:40.782100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7c60-e186-42c9-9f66-8a44b6204622 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:40.782385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7c60-e186-42c9-9f66-8a44b6204622 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:40.782665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e7c60-e186-42c9-9f66-8a44b6204622 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:40.783491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e7c60-e186-42c9-9f66-8a44b6204622 None admin] RBAC: Adding query filter params (interface=public, service_id=12dd44fb4e2e46bdbe3e61c47d27e1a2, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:40.783804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e7c60-e186-42c9-9f66-8a44b6204622 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:40.785179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e7c60-e186-42c9-9f66-8a44b6204622 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:40.790848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 463/924] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Jul 15 15:33:40 2022] GET /identity/v3/endpoints?service_id=12dd44fb4e2e46bdbe3e61c47d27e1a2&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:40.796406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:40.908161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ba0cc-ed2d-4f83-b067-7ecb160d2caf None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:40.935661 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:40.943631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf0cf9-970b-48de-a1b5-a32fef777951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:40.944306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf0cf9-970b-48de-a1b5-a32fef777951 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:40.945123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf0cf9-970b-48de-a1b5-a32fef777951 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:40.945403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf0cf9-970b-48de-a1b5-a32fef777951 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:40.946079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf0cf9-970b-48de-a1b5-a32fef777951 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:40.947491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf0cf9-970b-48de-a1b5-a32fef777951 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:40.954206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 462/925] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:33:40 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:33:41.182908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713852-bfb5-4031-ae01-5528807a0f3d None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.183249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713852-bfb5-4031-ae01-5528807a0f3d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.183479 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d713852-bfb5-4031-ae01-5528807a0f3d None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.185161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 464/926] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:41.190678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f37e-6b5a-46af-8f53-c4f06cdb54d7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.190983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f37e-6b5a-46af-8f53-c4f06cdb54d7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.191272 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f37e-6b5a-46af-8f53-c4f06cdb54d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.208357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2848f37e-6b5a-46af-8f53-c4f06cdb54d7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:41.209565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2848f37e-6b5a-46af-8f53-c4f06cdb54d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:41.283408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 463/927] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:41.302821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713852-bfb5-4031-ae01-5528807a0f3d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:41.399226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:41.413374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d713852-bfb5-4031-ae01-5528807a0f3d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:41.448831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e51812-ca44-4ca4-a7cf-7f08e35a9ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:41.449622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e51812-ca44-4ca4-a7cf-7f08e35a9ca2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.449911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e51812-ca44-4ca4-a7cf-7f08e35a9ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.450190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e51812-ca44-4ca4-a7cf-7f08e35a9ca2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.450842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e51812-ca44-4ca4-a7cf-7f08e35a9ca2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:41.452243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e51812-ca44-4ca4-a7cf-7f08e35a9ca2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a2e51812-ca44-4ca4-a7cf-7f08e35a9ca2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:41.465302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:41.466285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:41.467463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 465/928] 10.222.0.98 () {56 vars in 1132 bytes} [Fri Jul 15 15:33:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:41.477347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f37e-6b5a-46af-8f53-c4f06cdb54d7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:41.578224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f37e-6b5a-46af-8f53-c4f06cdb54d7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:41.601945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:41.613553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:41.614293 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.614583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.616008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.616008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:41.616008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:41.617907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:41.625380 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd780-df29-45f2-9b87-6a75dc27852b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.625670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd780-df29-45f2-9b87-6a75dc27852b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.625965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bd780-df29-45f2-9b87-6a75dc27852b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.627400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 464/929] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:41.627604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 466/930] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:41.629872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:41.633077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1dc9e-ebc1-4d4f-be84-21f21bcbbc53 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.633367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1dc9e-ebc1-4d4f-be84-21f21bcbbc53 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.633649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1dc9e-ebc1-4d4f-be84-21f21bcbbc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.654045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bef1dc9e-ebc1-4d4f-be84-21f21bcbbc53 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:41.656060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bef1dc9e-ebc1-4d4f-be84-21f21bcbbc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:41.739564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6acbb-2fe1-4214-ba9c-bfbd5eeba7ac None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:41.751458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 467/931] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:41.774680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1dc9e-ebc1-4d4f-be84-21f21bcbbc53 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:41.781901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:41.782578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.783271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.783607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.784322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:41.784636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:41.786786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:41.794098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 465/932] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:41.800214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:41.859753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:41.883694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:41.900229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1dc9e-ebc1-4d4f-be84-21f21bcbbc53 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:41.921881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023f7e4-0198-45e5-82b3-a4a0c52dbbe4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:41.937053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfc88e-f2eb-4234-9d6b-7a13f8b2cb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:41.938336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfc88e-f2eb-4234-9d6b-7a13f8b2cb21 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.938785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfc88e-f2eb-4234-9d6b-7a13f8b2cb21 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.939250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfc88e-f2eb-4234-9d6b-7a13f8b2cb21 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.953169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accfc88e-f2eb-4234-9d6b-7a13f8b2cb21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:41.965149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4848-8106-49bf-8f30-83443b03ce45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:41.965149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4848-8106-49bf-8f30-83443b03ce45 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:41.965149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4848-8106-49bf-8f30-83443b03ce45 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:41.965149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae4848-8106-49bf-8f30-83443b03ce45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:41.965473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae4848-8106-49bf-8f30-83443b03ce45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-accfc88e-f2eb-4234-9d6b-7a13f8b2cb21 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:41.965551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:41.966604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:33:41.967466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:41.968088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae4848-8106-49bf-8f30-83443b03ce45 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:41.968164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 468/933] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Jul 15 15:33:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:41.971182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfc88e-f2eb-4234-9d6b-7a13f8b2cb21 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:42.067529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 466/934] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:33:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:42.084472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfc88e-f2eb-4234-9d6b-7a13f8b2cb21 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:42.131977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:42.132972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:42.133405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:42.133860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:42.134792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:42.135258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:42.136981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:42.143856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:42.149187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 469/935] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Jul 15 15:33:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:42.154793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4848-8106-49bf-8f30-83443b03ce45 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:42.271826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae4848-8106-49bf-8f30-83443b03ce45 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:42.281203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:42.309401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:42.310807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:42.311096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:42.312171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:42.323432 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:42.325175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:42.344084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:42.345003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:42.345951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:42.345951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:42.345951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:42.345951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:42.348254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 467/936] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:42.354305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:42.458631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50965e59-2e0b-47ec-97e5-92c696e2158a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:42.494987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda138b-00fe-4d81-9d4a-023a03c11869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:42.495743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda138b-00fe-4d81-9d4a-023a03c11869 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:42.496030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda138b-00fe-4d81-9d4a-023a03c11869 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:42.496308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda138b-00fe-4d81-9d4a-023a03c11869 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:42.497070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda138b-00fe-4d81-9d4a-023a03c11869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:42.497381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda138b-00fe-4d81-9d4a-023a03c11869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:42.498761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda138b-00fe-4d81-9d4a-023a03c11869 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:42.511881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 470/937] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:42.517076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:42.617173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a6b50-02e8-474e-ad74-2b236436d6d2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:42.617503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a6b50-02e8-474e-ad74-2b236436d6d2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:42.617724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a6b50-02e8-474e-ad74-2b236436d6d2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:42.619881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 471/938] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:42.625072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571843-82df-4dfd-875d-4342d885d5e5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:42.625361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571843-82df-4dfd-875d-4342d885d5e5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:42.625641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f571843-82df-4dfd-875d-4342d885d5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:42.626291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543196d-9190-47a7-bd56-3ed46970e6f8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:42.643725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9f571843-82df-4dfd-875d-4342d885d5e5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:42.644911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9f571843-82df-4dfd-875d-4342d885d5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:42.672784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:42.674619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:42.675405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:42.675891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:42.689989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:42.692767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:42.705188 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:42.715013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:42.715969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:42.716875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:42.719448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 468/939] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:33:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:42.723758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 472/940] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:42.732309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:42.748781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f571843-82df-4dfd-875d-4342d885d5e5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:42.753179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:42.850512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce526f-193e-420b-b1f4-0deefd9c061c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:42.866834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f571843-82df-4dfd-875d-4342d885d5e5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:42.888142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:42.888846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:42.889363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:42.889618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:42.890752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:42.891158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:42.893074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:42.909639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:42.910394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:42.910733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:42.911087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:42.911936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:42.913535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:42.914444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 469/941] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Jul 15 15:33:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:42.915815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING keystone.server.flask.common [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 15 15:33:42.920263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:42.978803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.sql.core [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 15:33:42.978803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 15:33:42.978803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [parameters: {'id': '67696ad13f054951aca5556d37dda0f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 15:33:42.978803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87025) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67696ad13f054951aca5556d37dda0f0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67696ad13f054951aca5556d37dda0f0. Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:33:43.001344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result.read() Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 15 15:33:43.002211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application yield resource Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise self.value Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application session.commit() Jul 15 15:33:43.003133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.session.flush() Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application compat.raise_( Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 15:33:43.004041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 15:33:43.004916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result.read() Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:33:43.005745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [parameters: {'id': '67696ad13f054951aca5556d37dda0f0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.006616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:43.008812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 67696ad13f054951aca5556d37dda0f0. Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 473/942] 10.222.0.98 () {60 vars in 1140 bytes} [Fri Jul 15 15:33:42 2022] POST /identity/v3/projects => generated 262 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:43.010929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.013818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:43.039632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e0d1-8f8b-40a4-87dc-9c53e51aa9f0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.080252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:43.082890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7e65e-851b-41d1-adb4-2a354f7a690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.083961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7e65e-851b-41d1-adb4-2a354f7a690c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.084462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7e65e-851b-41d1-adb4-2a354f7a690c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.084981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7e65e-851b-41d1-adb4-2a354f7a690c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.096878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7e65e-851b-41d1-adb4-2a354f7a690c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.098643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7e65e-851b-41d1-adb4-2a354f7a690c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-65a7e65e-851b-41d1-adb4-2a354f7a690c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:43.117686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:43.118782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:43.119889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:43.119889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:43.119889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:43.119889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:43.121400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 470/943] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:43.125936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30967d-be56-442a-ae99-f903153b2f12 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.125936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30967d-be56-442a-ae99-f903153b2f12 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.125936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30967d-be56-442a-ae99-f903153b2f12 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.129481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 471/944] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:43.129481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30967d-be56-442a-ae99-f903153b2f12 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.148966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22055089-7c69-4de7-b445-eb8acbb4dd0e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.190319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2647a-12b1-4cee-a101-94592f13007c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.191504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2647a-12b1-4cee-a101-94592f13007c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.191787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2647a-12b1-4cee-a101-94592f13007c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.192062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2647a-12b1-4cee-a101-94592f13007c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.199494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2647a-12b1-4cee-a101-94592f13007c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.201142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2647a-12b1-4cee-a101-94592f13007c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e8e2647a-12b1-4cee-a101-94592f13007c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:43.215770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:43.216764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.217743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 474/945] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:43.218605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb93e7-707d-4a59-9e6b-36b55e1abae5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.218898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb93e7-707d-4a59-9e6b-36b55e1abae5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.219172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb93e7-707d-4a59-9e6b-36b55e1abae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.237506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-28cb93e7-707d-4a59-9e6b-36b55e1abae5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:43.239025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-28cb93e7-707d-4a59-9e6b-36b55e1abae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:43.246984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30967d-be56-442a-ae99-f903153b2f12 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.279495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:43.287261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.287958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.288271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.288550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.289269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:43.289576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.290953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.308619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 472/946] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:43.309706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.310435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 475/947] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:43.311329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb93e7-707d-4a59-9e6b-36b55e1abae5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.355423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:43.422097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8335a45-d6d0-44a4-ac5b-00e66c0f272f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.428630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb93e7-707d-4a59-9e6b-36b55e1abae5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.458280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.459034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.459368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.459658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.460590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:43.460909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.462513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.466656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.468055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.468537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.468991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.477240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.479141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.479717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 473/948] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:43.480510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:43.506958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:43.508557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:43.509420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:43.509420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:43.509420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:43.509420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:43.509420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:43.509420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:43.509420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:43.509420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:43.512882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 476/949] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:43.518915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.543986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:43.602299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac8a2-1849-4e6d-a67a-1630d8ef06ff None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.653504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.654326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.654614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.654896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.655611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.657205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.658851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:43.665928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb7d81-cdc0-49f7-8702-ef526e4c17c2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:43.673349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:43.677498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:43.678725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 474/950] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:43.681973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.713247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.713651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.713932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.714210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.715165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:43.715841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.717278 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.734040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 477/951] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:43.740144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.803220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df8f77-dca3-4f01-aed5-becfb5d60ff2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.819370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:43.849943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.851441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.851744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.852764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.853871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:43.854330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.856610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.859623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46be0e5-3f7f-48e8-ae43-92100c007f74 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:43.864043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 475/952] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:43.869900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:43.909241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebe5c4-4362-473c-b3c4-f7b6a20e7ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:43.910101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebe5c4-4362-473c-b3c4-f7b6a20e7ed3 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:43.910392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebe5c4-4362-473c-b3c4-f7b6a20e7ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:43.910675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebe5c4-4362-473c-b3c4-f7b6a20e7ed3 None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/a1daa10e8ac5455cbf98f02971639237/roles/973b62e0cb57414ca5df0f3a32e38887` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:43.914652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebe5c4-4362-473c-b3c4-f7b6a20e7ed3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=a1daa10e8ac5455cbf98f02971639237, role_id=973b62e0cb57414ca5df0f3a32e38887)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:43.916654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebe5c4-4362-473c-b3c4-f7b6a20e7ed3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:43.976573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5e050-7b3e-41ca-b3c0-bba2da7b5a81 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:44.012089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:44.012762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:44.013051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:44.013329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:44.014183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:44.014491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:44.015918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:44.024755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 476/953] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:44.030954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:44.047777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 478/954] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:33:43 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/a1daa10e8ac5455cbf98f02971639237/roles/973b62e0cb57414ca5df0f3a32e38887 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:33:44.093117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:44.160686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26b2fe-bed0-42a3-9348-b07bfab6dfdc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:44.200424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b7761-efe9-44f8-ad51-a3bbfa930634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:44.201209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b7761-efe9-44f8-ad51-a3bbfa930634 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:44.201555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b7761-efe9-44f8-ad51-a3bbfa930634 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:44.201838 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b7761-efe9-44f8-ad51-a3bbfa930634 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:44.202583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b7761-efe9-44f8-ad51-a3bbfa930634 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:44.204116 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b7761-efe9-44f8-ad51-a3bbfa930634 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:44.247781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357a95a-c112-4f38-94f9-6699591cf7c8 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:44.248233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357a95a-c112-4f38-94f9-6699591cf7c8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:44.248520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357a95a-c112-4f38-94f9-6699591cf7c8 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:44.250053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 479/955] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:44.255168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996383d4-ec00-4f10-96e1-7b904882ee88 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:44.258623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996383d4-ec00-4f10-96e1-7b904882ee88 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:44.258623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996383d4-ec00-4f10-96e1-7b904882ee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:44.280681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-996383d4-ec00-4f10-96e1-7b904882ee88 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:44.283235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-996383d4-ec00-4f10-96e1-7b904882ee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:44.291472 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 477/956] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:33:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:33:44.298956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:44.377058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 480/957] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:44.403575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b7761-efe9-44f8-ad51-a3bbfa930634 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:44.515028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:44.521763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b7761-efe9-44f8-ad51-a3bbfa930634 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:44.561884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:44.562771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:44.563058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:44.563353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:44.564009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:44.565478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:44.578592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:44.579693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:44.580675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 478/958] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:44.588058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996383d4-ec00-4f10-96e1-7b904882ee88 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:44.696289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996383d4-ec00-4f10-96e1-7b904882ee88 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:44.732055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8ada6-87ae-4015-85e7-0f3104b74722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:44.732729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8ada6-87ae-4015-85e7-0f3104b74722 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:44.733011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8ada6-87ae-4015-85e7-0f3104b74722 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:44.733289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8ada6-87ae-4015-85e7-0f3104b74722 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:44.733991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8ada6-87ae-4015-85e7-0f3104b74722 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:44.734298 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8ada6-87ae-4015-85e7-0f3104b74722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:44.735694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8ada6-87ae-4015-85e7-0f3104b74722 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:44.741161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 481/959] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:44.746964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:44.856087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091ccef-2575-49b8-92af-97c299486964 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:44.891376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62dd0c-351c-426a-8cb9-733d12eedf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:44.892294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62dd0c-351c-426a-8cb9-733d12eedf97 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:44.892747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62dd0c-351c-426a-8cb9-733d12eedf97 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:44.893173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62dd0c-351c-426a-8cb9-733d12eedf97 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:44.894091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62dd0c-351c-426a-8cb9-733d12eedf97 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:44.894548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62dd0c-351c-426a-8cb9-733d12eedf97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:44.896109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62dd0c-351c-426a-8cb9-733d12eedf97 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:44.907431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 479/960] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:45.694455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d7964-e1a7-43b1-8562-038b9f462272 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:45.694966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d7964-e1a7-43b1-8562-038b9f462272 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:45.694966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d7964-e1a7-43b1-8562-038b9f462272 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:45.696676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 482/961] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:45.701923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eada5b9-e1b9-40eb-9248-d2539a7cd999 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:45.702330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eada5b9-e1b9-40eb-9248-d2539a7cd999 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:45.702491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eada5b9-e1b9-40eb-9248-d2539a7cd999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:45.719760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5eada5b9-e1b9-40eb-9248-d2539a7cd999 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:45.720960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5eada5b9-e1b9-40eb-9248-d2539a7cd999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:45.740799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:45.793887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 480/962] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:45.838498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d7964-e1a7-43b1-8562-038b9f462272 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:45.847523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:45.958950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d7964-e1a7-43b1-8562-038b9f462272 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:45.995050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f867-c4ed-431d-b67d-a10dfb075d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:45.995680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f867-c4ed-431d-b67d-a10dfb075d33 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:45.995970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f867-c4ed-431d-b67d-a10dfb075d33 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:45.996253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f867-c4ed-431d-b67d-a10dfb075d33 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:45.996960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7f867-c4ed-431d-b67d-a10dfb075d33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:45.998392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7f867-c4ed-431d-b67d-a10dfb075d33 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:46.047167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.sql.core [None req-f2f7f867-c4ed-431d-b67d-a10dfb075d33 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 15:33:46.047167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 15:33:46.047167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [parameters: {'user_id': 'e5504c19afd946cc8c7b22a13b31ab50', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 15:33:46.047167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87025) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f2f7f867-c4ed-431d-b67d-a10dfb075d33 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result.read() Jul 15 15:33:46.055040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:33:46.055996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application yield resource Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise self.value Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application session.commit() Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 15:33:46.057149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.session.flush() Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application compat.raise_( Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 15:33:46.058486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:33:46.059406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result.read() Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:33:46.060234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e5504c19afd946cc8c7b22a13b31ab50', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:46.061733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 15 15:33:46.063180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 15 15:33:46.068801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 15 15:33:46.068801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 15 15:33:46.068801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 15:33:46.068801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 15:33:46.068801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 15:33:46.068801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:46.068801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 483/963] 10.222.0.98 () {60 vars in 1131 bytes} [Fri Jul 15 15:33:45 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:46.069333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eada5b9-e1b9-40eb-9248-d2539a7cd999 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:46.137910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9d6a5-372b-42ef-83fd-f2d20f1d2b55 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.138188 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9d6a5-372b-42ef-83fd-f2d20f1d2b55 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.138467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9d6a5-372b-42ef-83fd-f2d20f1d2b55 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.143786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 484/964] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:46.148530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397660b-fbd7-4f34-8f7d-fdabd956d4de None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.148689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397660b-fbd7-4f34-8f7d-fdabd956d4de None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.148842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397660b-fbd7-4f34-8f7d-fdabd956d4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.167134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0397660b-fbd7-4f34-8f7d-fdabd956d4de None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:46.168696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0397660b-fbd7-4f34-8f7d-fdabd956d4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:46.176228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eada5b9-e1b9-40eb-9248-d2539a7cd999 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:46.216175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:46.216892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.217177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.217453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.225117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:46.227122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:46.240122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:46.243495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:46.244690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:33:46.245700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:46.249723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 481/965] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:46.253354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 485/966] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:46.253499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:46.268771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:46.276572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397660b-fbd7-4f34-8f7d-fdabd956d4de None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:46.379632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967422-2a7a-4838-9d1f-193c84047cf9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:46.394099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397660b-fbd7-4f34-8f7d-fdabd956d4de None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:46.427797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbbde4-5d49-449a-bd7a-10636a270e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:46.428486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbbde4-5d49-449a-bd7a-10636a270e4a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.429564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbbde4-5d49-449a-bd7a-10636a270e4a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.429850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbbde4-5d49-449a-bd7a-10636a270e4a None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.430761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbbde4-5d49-449a-bd7a-10636a270e4a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:46.431067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbbde4-5d49-449a-bd7a-10636a270e4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:46.432703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbbde4-5d49-449a-bd7a-10636a270e4a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:46.434273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bde76f-a81a-4189-8c6e-4e7df9a43871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:46.435268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde76f-a81a-4189-8c6e-4e7df9a43871 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.435781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde76f-a81a-4189-8c6e-4e7df9a43871 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.436204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bde76f-a81a-4189-8c6e-4e7df9a43871 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.447723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bde76f-a81a-4189-8c6e-4e7df9a43871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c5bde76f-a81a-4189-8c6e-4e7df9a43871 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:46.456266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:46.457433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:33:46.458478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:46.465787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 486/967] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:46.467287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f4ac-ffdc-414d-bc1d-95626eb3e4bf None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.467557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f4ac-ffdc-414d-bc1d-95626eb3e4bf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.467838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f4ac-ffdc-414d-bc1d-95626eb3e4bf None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.469294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 487/968] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:46.469866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6f4ac-ffdc-414d-bc1d-95626eb3e4bf None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:46.470392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 482/969] 10.222.0.98 () {56 vars in 1101 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:46.475506 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118a6f6-7477-4b10-b2c9-b49bf4ab9a9d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.476373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118a6f6-7477-4b10-b2c9-b49bf4ab9a9d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.476678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118a6f6-7477-4b10-b2c9-b49bf4ab9a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.499515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4118a6f6-7477-4b10-b2c9-b49bf4ab9a9d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:46.501450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4118a6f6-7477-4b10-b2c9-b49bf4ab9a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:46.547222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:46.595804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6f4ac-ffdc-414d-bc1d-95626eb3e4bf None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:46.603565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 483/970] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:46.626618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118a6f6-7477-4b10-b2c9-b49bf4ab9a9d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:46.629015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:46.641767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:46.642504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.642726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.643064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.643793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:46.644107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:46.645624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:46.655147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 488/971] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:46.660941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:46.736703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118a6f6-7477-4b10-b2c9-b49bf4ab9a9d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:46.785664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59f0bf-bc4c-4f72-aa92-31ae02e05b1c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:46.786167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:46.786831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.788719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.788719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.788719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:46.789729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:46.802650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:46.804505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:46.805347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 484/972] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:46.810875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:46.811636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:46.825197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:46.825989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.826296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.826554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.837631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:46.839074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:46.860296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:46.861205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:46.862076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:46.862076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:46.862076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:46.862076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:46.862076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 489/973] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:46.867651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:46.883646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:46.926514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0fbc5-e4ee-43f6-9b69-9b137474c29e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:46.962114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:46.963521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:46.964021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:46.964618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:46.965539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:46.965853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:46.967238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:46.977386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 485/974] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:46.977550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ab82e-2127-418c-89aa-7c4f532858cd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:46.979187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:47.013588 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77fb11-fad4-45dc-ba6c-25b8fdbc6289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:47.014595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77fb11-fad4-45dc-ba6c-25b8fdbc6289 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.015213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77fb11-fad4-45dc-ba6c-25b8fdbc6289 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.015680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77fb11-fad4-45dc-ba6c-25b8fdbc6289 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.016653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77fb11-fad4-45dc-ba6c-25b8fdbc6289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:47.017105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77fb11-fad4-45dc-ba6c-25b8fdbc6289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:47.018611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77fb11-fad4-45dc-ba6c-25b8fdbc6289 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:47.031949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 490/975] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:47.033526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64924-36c9-4e00-938d-bc0f5675b602 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.033817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64924-36c9-4e00-938d-bc0f5675b602 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.034098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64924-36c9-4e00-938d-bc0f5675b602 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.035332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 491/976] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:47.037121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c64924-36c9-4e00-938d-bc0f5675b602 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:47.069764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:47.085588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74207e-541b-4613-801d-e157c3a1a9ba None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:47.114161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:47.120428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bb281-9610-4a86-9bea-d8bc170b9675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:47.121089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bb281-9610-4a86-9bea-d8bc170b9675 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.121385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bb281-9610-4a86-9bea-d8bc170b9675 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.121667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bb281-9610-4a86-9bea-d8bc170b9675 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.127180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bb281-9610-4a86-9bea-d8bc170b9675 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:47.127180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bb281-9610-4a86-9bea-d8bc170b9675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:47.127180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bb281-9610-4a86-9bea-d8bc170b9675 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:47.129083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 486/977] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:47.130321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d63d2f-3f5f-4db8-85fd-8eeae6bf1eca None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.130616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d63d2f-3f5f-4db8-85fd-8eeae6bf1eca None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.130902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d63d2f-3f5f-4db8-85fd-8eeae6bf1eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.140406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c64924-36c9-4e00-938d-bc0f5675b602 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:47.151742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-41d63d2f-3f5f-4db8-85fd-8eeae6bf1eca None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:47.152932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-41d63d2f-3f5f-4db8-85fd-8eeae6bf1eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:47.175145 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:47.175989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.176272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.176545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.184426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:47.186022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:47.201667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:47.202634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 492/978] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:33:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:47.204190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:47.230997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 487/979] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:47.233075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d63d2f-3f5f-4db8-85fd-8eeae6bf1eca None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:47.301699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:47.310273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262a92-2d13-4aca-adbf-6cc3dcc563a0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:47.336451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d63d2f-3f5f-4db8-85fd-8eeae6bf1eca None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:47.344358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:47.345015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.345300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.345577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.346360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] RBAC: Adding query filter params (interface=admin, service_id=12dd44fb4e2e46bdbe3e61c47d27e1a2, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:47.346670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:47.348066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:47.353642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 493/980] 10.222.0.98 () {56 vars in 1249 bytes} [Fri Jul 15 15:33:47 2022] GET /identity/v3/endpoints?service_id=12dd44fb4e2e46bdbe3e61c47d27e1a2&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:47.354385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:47.364933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:47.375903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:47.376571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.376854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.377178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.378001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:47.378309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:47.379859 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:47.398301 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 488/981] 10.222.0.98 () {56 vars in 1149 bytes} [Fri Jul 15 15:33:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:47.399130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:47.455767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c1e71-92a3-4755-adb5-2e1a5a14f1c2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:47.489494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9564749-8f3f-45ac-9cde-b722663a1cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:47.490151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9564749-8f3f-45ac-9cde-b722663a1cd5 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.490556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9564749-8f3f-45ac-9cde-b722663a1cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.490843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9564749-8f3f-45ac-9cde-b722663a1cd5 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.492085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9564749-8f3f-45ac-9cde-b722663a1cd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:47.493581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9564749-8f3f-45ac-9cde-b722663a1cd5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:47.496983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a7f3-d7c2-4d4b-bef9-7afd804e33fe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:47.531771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad0111-02f2-48a4-97d3-89b8dd389c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:47.532433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad0111-02f2-48a4-97d3-89b8dd389c65 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.532721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad0111-02f2-48a4-97d3-89b8dd389c65 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.533019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad0111-02f2-48a4-97d3-89b8dd389c65 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.533667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad0111-02f2-48a4-97d3-89b8dd389c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:47.535039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad0111-02f2-48a4-97d3-89b8dd389c65 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:47.540920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 489/982] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:33:47 2022] GET /identity/v3/services => generated 1497 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:33:47.541690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad0111-02f2-48a4-97d3-89b8dd389c65 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:47.569464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 494/983] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:33:47 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:47.597076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:47.661017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad0111-02f2-48a4-97d3-89b8dd389c65 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:47.709228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c48a66-df90-47c5-b361-4f769d06b8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:47.710002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c48a66-df90-47c5-b361-4f769d06b8b6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:47.710422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c48a66-df90-47c5-b361-4f769d06b8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:47.710538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c48a66-df90-47c5-b361-4f769d06b8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:47.782112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c48a66-df90-47c5-b361-4f769d06b8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:47.783695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c48a66-df90-47c5-b361-4f769d06b8b6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:47.826666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:47.889045 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 490/984] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:47 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:33:47.890267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9564749-8f3f-45ac-9cde-b722663a1cd5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:47.909130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:47.993988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9564749-8f3f-45ac-9cde-b722663a1cd5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:48.028802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:48.029597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:48.029884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:48.030162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:48.041453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:48.042909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:48.060045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:48.060964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:48.068179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:48.068179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:48.068179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:48.068179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:48.068179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 495/985] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:48.068467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c48a66-df90-47c5-b361-4f769d06b8b6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:48.168123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c48a66-df90-47c5-b361-4f769d06b8b6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:48.202417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:48.203083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:48.203396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:48.203680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:48.204401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:48.204710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:48.206093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:48.222212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 491/986] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:48.223359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:48.295098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:48.331004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46bb3a-ed75-4905-9747-f74ff0daaba1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:48.364390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:48.365292 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:48.365565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:48.365842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:48.372505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:48.373948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:48.391804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:48.392701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:48.393651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:48.393651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:48.393651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:48.393651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:48.393651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:48.393651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:48.393651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:48.393651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:48.394621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 496/987] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:48.399244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:48.402164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:48.501089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e011ab-1268-4ec4-b725-38e643de8149 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:48.534528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753883df-4af1-4ad7-ab9a-36dfe02078b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:48.535191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753883df-4af1-4ad7-ab9a-36dfe02078b5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:48.535506 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753883df-4af1-4ad7-ab9a-36dfe02078b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:48.535782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753883df-4af1-4ad7-ab9a-36dfe02078b5 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:48.536554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753883df-4af1-4ad7-ab9a-36dfe02078b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:48.536861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753883df-4af1-4ad7-ab9a-36dfe02078b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:48.538312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753883df-4af1-4ad7-ab9a-36dfe02078b5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:48.554819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 492/988] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:48.556042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:48.657088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa902b4-fd84-45f9-ad65-b9d2e70e1960 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:48.687370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:48.695723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba6f90-5029-40c0-8170-b619ec0e4292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:48.698435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba6f90-5029-40c0-8170-b619ec0e4292 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:48.698435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba6f90-5029-40c0-8170-b619ec0e4292 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:48.698435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba6f90-5029-40c0-8170-b619ec0e4292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:48.698435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba6f90-5029-40c0-8170-b619ec0e4292 None admin] RBAC: Adding query filter params (role.id=973b62e0cb57414ca5df0f3a32e38887, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=a1daa10e8ac5455cbf98f02971639237) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:48.698435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba6f90-5029-40c0-8170-b619ec0e4292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:48.699481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba6f90-5029-40c0-8170-b619ec0e4292 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:48.711322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 497/989] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:48 2022] GET /identity/v3/role_assignments?user.id=a1daa10e8ac5455cbf98f02971639237&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=973b62e0cb57414ca5df0f3a32e38887 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:49.582834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6433af-08b5-462b-9e30-162cb1b8e648 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:49.583363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6433af-08b5-462b-9e30-162cb1b8e648 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:49.583442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6433af-08b5-462b-9e30-162cb1b8e648 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:49.584797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 493/990] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:49.590874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236828-f24a-4044-843f-21dbab2f968b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:49.591270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236828-f24a-4044-843f-21dbab2f968b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:49.591480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c236828-f24a-4044-843f-21dbab2f968b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:49.607900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7c236828-f24a-4044-843f-21dbab2f968b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:49.609085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7c236828-f24a-4044-843f-21dbab2f968b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:49.664629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16e72-d01e-4c35-b2c6-b5448e1041c8 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:49.664949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16e72-d01e-4c35-b2c6-b5448e1041c8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:49.665114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16e72-d01e-4c35-b2c6-b5448e1041c8 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:49.666478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 494/991] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:49.671155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefab4f-09b4-4440-8a97-701cbdef98ac None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:49.671487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefab4f-09b4-4440-8a97-701cbdef98ac None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:49.671774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefab4f-09b4-4440-8a97-701cbdef98ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:49.676302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 498/992] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:49.688473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9eefab4f-09b4-4440-8a97-701cbdef98ac None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:49.689651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9eefab4f-09b4-4440-8a97-701cbdef98ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:49.699329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c236828-f24a-4044-843f-21dbab2f968b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:49.753270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:49.765747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 495/993] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:49.788229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefab4f-09b4-4440-8a97-701cbdef98ac None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:49.802729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c236828-f24a-4044-843f-21dbab2f968b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:49.826159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:49.840235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:49.841144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:49.841432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:49.841721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:49.842452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:49.843881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:49.857837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:49.859092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:33:49.860160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:49.864025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 499/994] 10.222.0.98 () {56 vars in 1126 bytes} [Fri Jul 15 15:33:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:49.865319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:49.907636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefab4f-09b4-4440-8a97-701cbdef98ac None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:49.945766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:49.946487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:49.946781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:49.947061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:49.947742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:49.949131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:49.961033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:49.967193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:49.968068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 496/995] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:49.968860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:49.974047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803eb093-8b8d-4af8-97c8-6ec59fcde1dc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:50.012478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:50.013151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.013759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.014043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.014900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:50.015211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:50.016611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:50.021654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 500/996] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:50.026840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:50.027624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:50.071308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8139d72-751c-4283-aa54-fe091c488196 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:50.082106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:50.105977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:50.106646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.106937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.107221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.108049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:50.108359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:50.109741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:50.114743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 497/997] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:50.120229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:50.137760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2b78a-8304-4042-b1a7-ecb3e0831772 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:50.174424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:50.175189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.175502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.175779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.176590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:50.176890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:50.178352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:50.188483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 501/998] 10.222.0.98 () {56 vars in 1122 bytes} [Fri Jul 15 15:33:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:50.189246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:50.224309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c081c80-2312-4257-a900-a5ba1505e324 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:50.244518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:50.258777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:50.261632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.261632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.261632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.261632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] RBAC: Adding query filter params (type=compute) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:50.261632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:50.262482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:50.267496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 498/999] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:50.273686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:50.299552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122bd28-8eee-4bb3-9acd-523b3fd6d71c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:50.316592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:50.347034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21fa5c-54ce-4d32-86b6-fbdb51a1ea21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:50.347837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21fa5c-54ce-4d32-86b6-fbdb51a1ea21 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.348017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21fa5c-54ce-4d32-86b6-fbdb51a1ea21 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.348290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21fa5c-54ce-4d32-86b6-fbdb51a1ea21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.349195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21fa5c-54ce-4d32-86b6-fbdb51a1ea21 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:50.350979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21fa5c-54ce-4d32-86b6-fbdb51a1ea21 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:50.391534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38a562-586b-4e3f-b472-388e9f9a86bc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:50.426687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:50.427385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.427676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.427957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.428741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] RBAC: Adding query filter params (interface=public, service_id=519bec06d7a342ff915e074ecb2345fc, region_id=RegionOne) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:50.429649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:50.430979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:50.436112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 499/1000] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Jul 15 15:33:50 2022] GET /identity/v3/endpoints?service_id=519bec06d7a342ff915e074ecb2345fc&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:50.441551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:50.454806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 502/1001] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:33:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:33:50.509407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:50.564779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732ddec-0151-4e50-bd87-a4525b64f0ab None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:50.607249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2565c5-8bb5-4e25-98d9-30f5ddd81526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:50.607962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2565c5-8bb5-4e25-98d9-30f5ddd81526 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.608217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2565c5-8bb5-4e25-98d9-30f5ddd81526 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.608489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2565c5-8bb5-4e25-98d9-30f5ddd81526 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.609169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2565c5-8bb5-4e25-98d9-30f5ddd81526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:50.610546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2565c5-8bb5-4e25-98d9-30f5ddd81526 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:50.618976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 500/1002] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:33:50 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:33:50.780884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3febc68-3b2b-4d96-87d1-67cfb441f8c7 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.781365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3febc68-3b2b-4d96-87d1-67cfb441f8c7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.781365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3febc68-3b2b-4d96-87d1-67cfb441f8c7 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.782746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 503/1003] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:50.788904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac390b-b07f-4516-9271-cb32220757c0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:50.789211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac390b-b07f-4516-9271-cb32220757c0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:50.789497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac390b-b07f-4516-9271-cb32220757c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:50.807637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9eac390b-b07f-4516-9271-cb32220757c0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:50.809125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9eac390b-b07f-4516-9271-cb32220757c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:50.879357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 501/1004] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:50.901082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3febc68-3b2b-4d96-87d1-67cfb441f8c7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:51.031828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3febc68-3b2b-4d96-87d1-67cfb441f8c7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:51.066809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:51.068099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:51.068593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:51.069101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] PATH_INFO: `/v3/services/image` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:51.070489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:51.072100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:51.084119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:51.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:51.085950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:51.092453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 504/1005] 10.222.0.98 () {56 vars in 1105 bytes} [Fri Jul 15 15:33:50 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:51.094122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac390b-b07f-4516-9271-cb32220757c0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:51.136795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:51.212809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac390b-b07f-4516-9271-cb32220757c0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:51.249202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027da8c0-7bb8-47e9-8380-fce39554b129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:51.249871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027da8c0-7bb8-47e9-8380-fce39554b129 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:51.250155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027da8c0-7bb8-47e9-8380-fce39554b129 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:51.250430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027da8c0-7bb8-47e9-8380-fce39554b129 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:51.251134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027da8c0-7bb8-47e9-8380-fce39554b129 None admin] RBAC: Adding query filter params (name=image) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:51.251469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027da8c0-7bb8-47e9-8380-fce39554b129 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:51.252841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027da8c0-7bb8-47e9-8380-fce39554b129 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:51.258456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 502/1006] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:51.264045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:51.320996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:51.392002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22d833-f383-4316-9736-68575f2bab6f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:51.432289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c86f7-a662-4c2e-96ca-3f0e08c315e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:51.432955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c86f7-a662-4c2e-96ca-3f0e08c315e6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:51.433244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c86f7-a662-4c2e-96ca-3f0e08c315e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:51.433526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c86f7-a662-4c2e-96ca-3f0e08c315e6 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:51.434224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c86f7-a662-4c2e-96ca-3f0e08c315e6 None admin] RBAC: Adding query filter params (type=image) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:51.434536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c86f7-a662-4c2e-96ca-3f0e08c315e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:51.435930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c86f7-a662-4c2e-96ca-3f0e08c315e6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:51.447341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 505/1007] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:52.525251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408b673-4c96-4b54-88a3-dbf51b041651 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:52.525749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408b673-4c96-4b54-88a3-dbf51b041651 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:52.525860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408b673-4c96-4b54-88a3-dbf51b041651 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:52.527353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 503/1008] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:52.532643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde8d0b-d1db-47c6-b7cd-4315e0da199e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:52.533023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde8d0b-d1db-47c6-b7cd-4315e0da199e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:52.533204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde8d0b-d1db-47c6-b7cd-4315e0da199e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:52.549942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3dde8d0b-d1db-47c6-b7cd-4315e0da199e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:52.551160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3dde8d0b-d1db-47c6-b7cd-4315e0da199e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:52.620600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 506/1009] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:52.643214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408b673-4c96-4b54-88a3-dbf51b041651 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:52.653234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f0fa5-1df2-4304-bae2-cd8b0ced78ac None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:52.653787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f0fa5-1df2-4304-bae2-cd8b0ced78ac None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:52.654072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f0fa5-1df2-4304-bae2-cd8b0ced78ac None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:52.655503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 507/1010] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:52.660826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3ee2-8c0b-44a9-b79e-8f5becc611ac None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:52.661059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3ee2-8c0b-44a9-b79e-8f5becc611ac None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:52.661342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3ee2-8c0b-44a9-b79e-8f5becc611ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:52.677876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-744f3ee2-8c0b-44a9-b79e-8f5becc611ac None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:52.679585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-744f3ee2-8c0b-44a9-b79e-8f5becc611ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:52.706612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:52.748486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 508/1011] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:52.752984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408b673-4c96-4b54-88a3-dbf51b041651 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:52.755821 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:52.771345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3ee2-8c0b-44a9-b79e-8f5becc611ac None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:52.792592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cc88-15a8-40f6-b868-235b3fcb78d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:52.793676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cc88-15a8-40f6-b868-235b3fcb78d7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:52.794147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cc88-15a8-40f6-b868-235b3fcb78d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:52.794682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cc88-15a8-40f6-b868-235b3fcb78d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:52.796774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295cc88-15a8-40f6-b868-235b3fcb78d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:52.798225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295cc88-15a8-40f6-b868-235b3fcb78d7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:52.799869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 504/1012] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:52.805833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cc88-15a8-40f6-b868-235b3fcb78d7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:52.877067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3ee2-8c0b-44a9-b79e-8f5becc611ac None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:52.910350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cc88-15a8-40f6-b868-235b3fcb78d7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:52.930438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:52.930438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:52.930438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:52.930438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] PATH_INFO: `/v3/services/compute` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:52.930818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:52.930818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:52.936320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:52.937300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:52.938132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 509/1013] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:52.943441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:52.951341 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75056045-c01e-40e7-b8c7-9216bddec3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:52.952028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75056045-c01e-40e7-b8c7-9216bddec3db None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:52.952320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75056045-c01e-40e7-b8c7-9216bddec3db None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:52.952608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75056045-c01e-40e7-b8c7-9216bddec3db None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:52.953321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75056045-c01e-40e7-b8c7-9216bddec3db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:52.954773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75056045-c01e-40e7-b8c7-9216bddec3db None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:52.973612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:53.055336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 505/1014] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:33:52 2022] POST /identity/v3/projects => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:53.074877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f113d4-da5a-40d9-93cc-622954e47cfe None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:53.113441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:53.114129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:53.114415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:53.114693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:53.115431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:53.115740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:53.117746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:53.123513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 510/1015] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:53.129359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75056045-c01e-40e7-b8c7-9216bddec3db None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:53.157128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:53.245734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75056045-c01e-40e7-b8c7-9216bddec3db None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:53.281895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823d1c6-0809-49f6-8b92-ad36718e82b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:53.282564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823d1c6-0809-49f6-8b92-ad36718e82b5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:53.282850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823d1c6-0809-49f6-8b92-ad36718e82b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:53.283181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823d1c6-0809-49f6-8b92-ad36718e82b5 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:53.283955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823d1c6-0809-49f6-8b92-ad36718e82b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:53.284260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823d1c6-0809-49f6-8b92-ad36718e82b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:53.285634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823d1c6-0809-49f6-8b92-ad36718e82b5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:53.291147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 506/1016] 10.222.0.98 () {56 vars in 1112 bytes} [Fri Jul 15 15:33:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:53.297183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:53.409151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:53.420296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa5c65-d29f-4f08-b07a-32514ab8f042 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:53.453693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62bc0e-3ebd-42cc-a818-a8899012e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:53.454892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62bc0e-3ebd-42cc-a818-a8899012e1e7 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:53.456113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62bc0e-3ebd-42cc-a818-a8899012e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:53.456387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62bc0e-3ebd-42cc-a818-a8899012e1e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:53.457141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62bc0e-3ebd-42cc-a818-a8899012e1e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:53.458671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62bc0e-3ebd-42cc-a818-a8899012e1e7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:53.494184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266ab30-f64a-4d9c-b5fe-90bbc4d83be5 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:53.494599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266ab30-f64a-4d9c-b5fe-90bbc4d83be5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:53.494944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266ab30-f64a-4d9c-b5fe-90bbc4d83be5 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:53.496468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 507/1017] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:53.501440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c88d0-7367-4d08-8da0-efc19891984c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:53.507409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c88d0-7367-4d08-8da0-efc19891984c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:53.507409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c88d0-7367-4d08-8da0-efc19891984c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:53.519080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ee8c88d0-7367-4d08-8da0-efc19891984c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:53.520284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ee8c88d0-7367-4d08-8da0-efc19891984c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:53.534232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 511/1018] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:33:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:33:53.612903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 508/1019] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:53.636243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62bc0e-3ebd-42cc-a818-a8899012e1e7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:53.747440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62bc0e-3ebd-42cc-a818-a8899012e1e7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:53.786997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade1d57-416e-45cb-88e9-9b6eedf8668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:53.787826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade1d57-416e-45cb-88e9-9b6eedf8668f None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:53.788135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade1d57-416e-45cb-88e9-9b6eedf8668f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:53.788431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade1d57-416e-45cb-88e9-9b6eedf8668f None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:53.789164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade1d57-416e-45cb-88e9-9b6eedf8668f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:53.790610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade1d57-416e-45cb-88e9-9b6eedf8668f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:53.888942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 512/1020] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:33:53 2022] POST /identity/v3/services => generated 242 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:33:55.126563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab55f9-564a-41e7-a3c2-7d07150a4dca None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.127085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab55f9-564a-41e7-a3c2-7d07150a4dca None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.127167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab55f9-564a-41e7-a3c2-7d07150a4dca None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:55.128591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 509/1021] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:55.133741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e561d-2966-422d-bd9d-c5a1c26bc959 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.134181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e561d-2966-422d-bd9d-c5a1c26bc959 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.134338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e561d-2966-422d-bd9d-c5a1c26bc959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:55.146255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:55.151826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-381e561d-2966-422d-bd9d-c5a1c26bc959 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:55.153015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-381e561d-2966-422d-bd9d-c5a1c26bc959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:55.227663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 513/1022] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:55.251653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab55f9-564a-41e7-a3c2-7d07150a4dca None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:55.266499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:55.357590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab55f9-564a-41e7-a3c2-7d07150a4dca None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:55.393602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ba59e-c9dd-4545-a322-48bc24e4a208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:55.394765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ba59e-c9dd-4545-a322-48bc24e4a208 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.395278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ba59e-c9dd-4545-a322-48bc24e4a208 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.395804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ba59e-c9dd-4545-a322-48bc24e4a208 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:55.397203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ba59e-c9dd-4545-a322-48bc24e4a208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:55.399603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ba59e-c9dd-4545-a322-48bc24e4a208 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:55.401830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 510/1023] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:55.407521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e561d-2966-422d-bd9d-c5a1c26bc959 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:55.513157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e561d-2966-422d-bd9d-c5a1c26bc959 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:55.542757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:55.548259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75ed7f-3b5b-45d7-9cb5-29980cbeddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:55.548935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ed7f-3b5b-45d7-9cb5-29980cbeddca None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.549221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ed7f-3b5b-45d7-9cb5-29980cbeddca None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.549500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ed7f-3b5b-45d7-9cb5-29980cbeddca None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:55.550218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75ed7f-3b5b-45d7-9cb5-29980cbeddca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:55.551934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75ed7f-3b5b-45d7-9cb5-29980cbeddca None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:55.667411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 514/1024] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:33:55 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:33:55.715341 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6449f7-71de-4fe1-b690-807301badec4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.715648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6449f7-71de-4fe1-b690-807301badec4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.715946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6449f7-71de-4fe1-b690-807301badec4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:55.717551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 511/1025] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:55.723177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fd906-52f3-4c69-8118-b8d2220712de None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.723504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fd906-52f3-4c69-8118-b8d2220712de None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.723788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fd906-52f3-4c69-8118-b8d2220712de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:55.742100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1c2fd906-52f3-4c69-8118-b8d2220712de None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:55.743398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1c2fd906-52f3-4c69-8118-b8d2220712de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:55.821119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 515/1026] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:55.843976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6449f7-71de-4fe1-b690-807301badec4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:55.927462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af1bb7-b20f-47cd-9b2d-e483d0527bbb None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.927682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af1bb7-b20f-47cd-9b2d-e483d0527bbb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.927962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af1bb7-b20f-47cd-9b2d-e483d0527bbb None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:55.929435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 516/1027] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:55.931018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:55.934568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ed519-74aa-4ba4-b409-4462564779d0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.934934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ed519-74aa-4ba4-b409-4462564779d0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.935283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ed519-74aa-4ba4-b409-4462564779d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:55.952162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6449f7-71de-4fe1-b690-807301badec4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:55.955914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-584ed519-74aa-4ba4-b409-4462564779d0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:55.957163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-584ed519-74aa-4ba4-b409-4462564779d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:55.987596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0de666-fcbe-43ae-bee6-65e281aa77b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:55.988420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0de666-fcbe-43ae-bee6-65e281aa77b3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:55.988879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0de666-fcbe-43ae-bee6-65e281aa77b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:55.989015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0de666-fcbe-43ae-bee6-65e281aa77b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.000043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0de666-fcbe-43ae-bee6-65e281aa77b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ad0de666-fcbe-43ae-bee6-65e281aa77b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:56.009083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:56.014614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:56.015767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 512/1028] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Jul 15 15:33:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:56.016669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0de666-fcbe-43ae-bee6-65e281aa77b3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.034950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 517/1029] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:33:56.057339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ed519-74aa-4ba4-b409-4462564779d0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.108966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:56.123469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0de666-fcbe-43ae-bee6-65e281aa77b3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.160556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.161213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ed519-74aa-4ba4-b409-4462564779d0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.161550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.161550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.161860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.162490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:56.162804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.164275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.177765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 513/1030] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:56.179832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.182553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:56.202405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.203208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.203571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.203851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] PATH_INFO: `/v3/services/image` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.204507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.205896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:56.219222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:56.220367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:56.221678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 518/1031] 10.222.0.98 () {56 vars in 1105 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:56.226285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.295898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e07c2-52e6-4dfe-aa06-a548413ae7af None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.332990 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.334134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.334423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.334707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.337777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c7411-637d-4f75-8111-47d08920428d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.346812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.348325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.354229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:56.367504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:56.368438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:56.369375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:56.369375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:56.369375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:56.369375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:56.371449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 514/1032] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:56.374554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.381891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.381891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.381891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.382170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.384406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] RBAC: Adding query filter params (name=image) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:56.384738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.386122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.396079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 519/1033] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:56.397227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.441315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:56.488238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc27ae-bdf5-425c-a694-8894ddfbc132 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.506843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40368c47-3edb-44be-b549-97a797143d4b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.524668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.525617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.525906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.526184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.526900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:56.527205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.529291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.545619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 515/1034] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:56.545799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.545799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.545799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.545799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.546127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] RBAC: Adding query filter params (type=image) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:56.546127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.546127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.547518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.551136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 520/1035] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:56.556997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.598413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:56.656311 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8dfc6-ea34-43d6-a969-ca883db32d7c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.659643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b872973-9348-46f4-85d7-601b1d07f731 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.671382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:56.693873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.694572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.694907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.695191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.695477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.696134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.696424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.696705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.697491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] RBAC: Adding query filter params (interface=public, service_id=d2c05fe55e9246c79a8097dbef7ed3a0, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:56.697800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.699173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.702875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.704643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.704751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 521/1036] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/endpoints?service_id=d2c05fe55e9246c79a8097dbef7ed3a0&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:56.709960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:33:56.719131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:56.720168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 516/1037] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:56.726948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:56.826423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee6244-cf4e-49cd-a066-a255026e003e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.829713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:56.836762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540ecc0-bb8f-4fe1-806b-e06b115efa75 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:56.865524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d577336-978c-4fb7-8e9b-3ebb7a3ae158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.866296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d577336-978c-4fb7-8e9b-3ebb7a3ae158 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.866471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d577336-978c-4fb7-8e9b-3ebb7a3ae158 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.867314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d577336-978c-4fb7-8e9b-3ebb7a3ae158 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.868007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d577336-978c-4fb7-8e9b-3ebb7a3ae158 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.869397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d577336-978c-4fb7-8e9b-3ebb7a3ae158 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.875475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:56.876153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:56.876442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:56.887582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:56.887831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 522/1038] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:33:56.887931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:56.887931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:56.887931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:56.906255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 517/1039] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:56.912189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d577336-978c-4fb7-8e9b-3ebb7a3ae158 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:57.020936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d577336-978c-4fb7-8e9b-3ebb7a3ae158 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:57.057551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba1f34-72f8-4880-8056-3ed4a2ca3572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:57.058199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1f34-72f8-4880-8056-3ed4a2ca3572 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:57.058488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1f34-72f8-4880-8056-3ed4a2ca3572 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:57.058766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba1f34-72f8-4880-8056-3ed4a2ca3572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:57.094518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:57.126971 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba1f34-72f8-4880-8056-3ed4a2ca3572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:57.128449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba1f34-72f8-4880-8056-3ed4a2ca3572 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:57.234285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 523/1040] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:33:57.236668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:57.270716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:57.340892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fba26-42e2-4dfb-9ce1-079b84795885 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:57.377017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:57.377810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:57.378094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:57.378368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:57.388795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:57.390260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:33:57.408875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:57.409813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:57.416353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:33:57.416353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:33:57.416353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:33:57.416353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:57.416353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 518/1041] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:57.416626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba1f34-72f8-4880-8056-3ed4a2ca3572 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:57.485557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:57.519202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba1f34-72f8-4880-8056-3ed4a2ca3572 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:57.555842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:57.556501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:57.556780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:57.557051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:57.557755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:57.558059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:57.559441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:57.574984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 524/1042] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:57.576418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:57.660068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:57.676122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e5530-fdc2-48e9-a5b3-8031afcbfa36 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:57.709858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091dcbb-a54d-4534-9f12-992d298e86dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:57.711170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091dcbb-a54d-4534-9f12-992d298e86dc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:57.711481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091dcbb-a54d-4534-9f12-992d298e86dc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:57.711756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091dcbb-a54d-4534-9f12-992d298e86dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:57.718353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091dcbb-a54d-4534-9f12-992d298e86dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:57.719854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091dcbb-a54d-4534-9f12-992d298e86dc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6091dcbb-a54d-4534-9f12-992d298e86dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:33:57.733299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:57.734165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:57.735113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 519/1043] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:57.741339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:57.771060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af71b-4178-4611-9ade-d1cb27e860a7 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:57.771394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af71b-4178-4611-9ade-d1cb27e860a7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:57.771665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2af71b-4178-4611-9ade-d1cb27e860a7 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:57.773210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 520/1044] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:33:57.777863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9fc0-96a6-41e0-8981-2fe016cf750a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:57.778159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9fc0-96a6-41e0-8981-2fe016cf750a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:57.778440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9fc0-96a6-41e0-8981-2fe016cf750a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:57.795478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-391d9fc0-96a6-41e0-8981-2fe016cf750a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:57.796685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-391d9fc0-96a6-41e0-8981-2fe016cf750a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:57.850002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53bdbe-6365-43f6-9248-3e3a48b1d302 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:57.869575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 521/1045] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:57.878308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:57.886687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:57.887449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:57.887801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:57.888143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:57.888990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:57.889364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:57.890990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:57.893657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d9fc0-96a6-41e0-8981-2fe016cf750a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:57.903209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 525/1046] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:57.909320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:57.979530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:58.009289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d9fc0-96a6-41e0-8981-2fe016cf750a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:58.016181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb429e-9bc2-4d56-841b-0289dfb37bac None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:58.052178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670b237-7f2e-4c51-b966-3a40a2e98975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:58.052387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:58.052938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:58.053042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670b237-7f2e-4c51-b966-3a40a2e98975 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:58.053221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:58.053317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670b237-7f2e-4c51-b966-3a40a2e98975 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:58.053483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:58.053557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670b237-7f2e-4c51-b966-3a40a2e98975 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:58.054805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] RBAC: Adding query filter params (role.id=973b62e0cb57414ca5df0f3a32e38887, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=bf670978cc2e4c529093a191bc26f885) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:58.055114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:58.056574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:58.067332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 526/1047] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:57 2022] GET /identity/v3/role_assignments?user.id=bf670978cc2e4c529093a191bc26f885&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=973b62e0cb57414ca5df0f3a32e38887 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:58.067517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670b237-7f2e-4c51-b966-3a40a2e98975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-8670b237-7f2e-4c51-b966-3a40a2e98975 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:33:58.072651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:58.073630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:33:58.074473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 522/1048] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:33:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:33:58.080157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:58.139493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:58.188721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08f503-c586-4e8b-bff6-eb75017359c1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:58.227335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:58.227335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:58.227335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:58.227696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:58.228363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:58.229764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:58.230024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:58.238483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 527/1049] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:58.244079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670b237-7f2e-4c51-b966-3a40a2e98975 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:58.344500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670b237-7f2e-4c51-b966-3a40a2e98975 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:58.382068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e3f86-01ce-4983-8c98-2d81c64a9dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:58.382068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3f86-01ce-4983-8c98-2d81c64a9dbc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:58.382068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3f86-01ce-4983-8c98-2d81c64a9dbc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:58.382068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3f86-01ce-4983-8c98-2d81c64a9dbc None admin] PATH_INFO: `/v3/users/d398c1480db34df1973af88d2250c910` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:58.384963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:58.396973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e3f86-01ce-4983-8c98-2d81c64a9dbc None admin] RBAC: Authorizing `identity:get_user(user_id=d398c1480db34df1973af88d2250c910)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:58.399076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e3f86-01ce-4983-8c98-2d81c64a9dbc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:58.400993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 523/1050] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:33:58 2022] GET /identity/v3/users/d398c1480db34df1973af88d2250c910 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:58.406874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:58.508304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8a1aa-0376-4050-9f1a-3e4ff60c0c05 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:58.523889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:58.544700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fc128-9e3d-4545-bd7d-26d1ee8ee100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:58.545349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fc128-9e3d-4545-bd7d-26d1ee8ee100 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:58.545632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fc128-9e3d-4545-bd7d-26d1ee8ee100 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:58.545905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fc128-9e3d-4545-bd7d-26d1ee8ee100 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:58.613542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fc128-9e3d-4545-bd7d-26d1ee8ee100 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:58.614986 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fc128-9e3d-4545-bd7d-26d1ee8ee100 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:58.716442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 528/1051] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:33:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:33:58.717659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e3f86-01ce-4983-8c98-2d81c64a9dbc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:58.816538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e3f86-01ce-4983-8c98-2d81c64a9dbc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:58.850262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ab325-ba75-4bbd-9ef8-a51881590849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:58.851182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab325-ba75-4bbd-9ef8-a51881590849 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:58.851563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab325-ba75-4bbd-9ef8-a51881590849 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:58.851845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab325-ba75-4bbd-9ef8-a51881590849 None admin] PATH_INFO: `/v3/projects/630f17f115b441928ebf9783eb54b761` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:58.853022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ab325-ba75-4bbd-9ef8-a51881590849 None admin] RBAC: Authorizing `identity:get_project(project_id=630f17f115b441928ebf9783eb54b761)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:58.854457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ab325-ba75-4bbd-9ef8-a51881590849 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:58.856325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 524/1052] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:33:58 2022] GET /identity/v3/projects/630f17f115b441928ebf9783eb54b761 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:58.862163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fc128-9e3d-4545-bd7d-26d1ee8ee100 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:58.878207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:58.923222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445957fd-2bb4-47da-b436-75c9cc89141d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:58.923537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445957fd-2bb4-47da-b436-75c9cc89141d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:58.923817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445957fd-2bb4-47da-b436-75c9cc89141d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:58.925187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 525/1053] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:33:58.930327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4b1c-0c49-4a65-b1d9-91e9e1ffec49 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:58.930926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4b1c-0c49-4a65-b1d9-91e9e1ffec49 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:58.931381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4b1c-0c49-4a65-b1d9-91e9e1ffec49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:58.953262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d1af4b1c-0c49-4a65-b1d9-91e9e1ffec49 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:58.954963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d1af4b1c-0c49-4a65-b1d9-91e9e1ffec49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:58.971825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fc128-9e3d-4545-bd7d-26d1ee8ee100 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:59.006600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:59.007565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:59.007851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:59.008133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:59.008953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=630f17f115b441928ebf9783eb54b761, user.id=d398c1480db34df1973af88d2250c910) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:59.009259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:59.010676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:59.016585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:59.027536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 529/1054] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:33:58 2022] GET /identity/v3/role_assignments?user.id=d398c1480db34df1973af88d2250c910&scope.project.id=630f17f115b441928ebf9783eb54b761&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:33:59.031554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 526/1055] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:33:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:33:59.052086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:59.142373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:59.167584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0dd05-ac5f-4fde-a5dc-617c073ee629 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:59.204262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:59.205049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:59.205332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:59.205606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] PATH_INFO: `/v3/services/image` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:59.206251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:59.207641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:33:59.219585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:33:59.220575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:33:59.226900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 530/1056] 10.222.0.98 () {56 vars in 1105 bytes} [Fri Jul 15 15:33:59 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:33:59.229461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af4b1c-0c49-4a65-b1d9-91e9e1ffec49 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:59.333900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af4b1c-0c49-4a65-b1d9-91e9e1ffec49 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:59.370518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:59.371357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:59.372593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:59.372896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:59.373628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] RBAC: Adding query filter params (name=image) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:59.374526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:59.375957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:59.385348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 527/1057] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:59.391441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:59.498042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cadd3-94a1-4549-9c3a-c1badde3d880 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:59.531950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0ed4b-fde0-45fb-a3fc-4084d8b7d77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:59.532611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0ed4b-fde0-45fb-a3fc-4084d8b7d77c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:59.532895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0ed4b-fde0-45fb-a3fc-4084d8b7d77c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:59.533171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0ed4b-fde0-45fb-a3fc-4084d8b7d77c None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:59.533864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0ed4b-fde0-45fb-a3fc-4084d8b7d77c None admin] RBAC: Adding query filter params (type=image) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:33:59.534168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0ed4b-fde0-45fb-a3fc-4084d8b7d77c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:59.535552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0ed4b-fde0-45fb-a3fc-4084d8b7d77c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:59.545013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 531/1058] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:33:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:33:59.546502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:59.604408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:59.648842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c84a2-559c-4b58-9061-878c9aa08a70 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:59.684312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dbc88-8119-4016-8cde-5da21fb8673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:59.684901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dbc88-8119-4016-8cde-5da21fb8673d None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:59.685190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dbc88-8119-4016-8cde-5da21fb8673d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:59.685474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dbc88-8119-4016-8cde-5da21fb8673d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:59.686217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1dbc88-8119-4016-8cde-5da21fb8673d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:59.687644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1dbc88-8119-4016-8cde-5da21fb8673d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:59.751390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 528/1059] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:33:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:34:00.093052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0609e-fa38-4e0a-92f9-d16bf5ab1054 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:00.093410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0609e-fa38-4e0a-92f9-d16bf5ab1054 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:00.093630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0609e-fa38-4e0a-92f9-d16bf5ab1054 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:00.095159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 532/1060] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:00.100267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09132b4-da42-46c6-9e7c-6e915fc3e4dc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:00.100568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09132b4-da42-46c6-9e7c-6e915fc3e4dc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:00.100853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09132b4-da42-46c6-9e7c-6e915fc3e4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:00.119273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a09132b4-da42-46c6-9e7c-6e915fc3e4dc None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:00.120539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a09132b4-da42-46c6-9e7c-6e915fc3e4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:00.199727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 529/1061] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:00.221930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0609e-fa38-4e0a-92f9-d16bf5ab1054 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:00.231823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:00.326105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0609e-fa38-4e0a-92f9-d16bf5ab1054 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:00.358701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9ed09-73f0-4391-b178-6771a884e07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:00.359457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9ed09-73f0-4391-b178-6771a884e07d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:00.359742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9ed09-73f0-4391-b178-6771a884e07d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:00.360016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9ed09-73f0-4391-b178-6771a884e07d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:00.370969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9ed09-73f0-4391-b178-6771a884e07d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-05d9ed09-73f0-4391-b178-6771a884e07d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:00.379737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:00.380698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:00.381533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 533/1062] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Jul 15 15:34:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:00.387326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09132b4-da42-46c6-9e7c-6e915fc3e4dc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:00.425108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:00.492591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09132b4-da42-46c6-9e7c-6e915fc3e4dc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:00.526611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:00.527284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:00.527592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:00.527867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:00.528609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:00.529173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:00.530557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:00.539465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 530/1063] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Jul 15 15:34:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:00.545033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9ed09-73f0-4391-b178-6771a884e07d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:00.610223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:00.644270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9ed09-73f0-4391-b178-6771a884e07d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:00.678418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:00.679203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:00.680078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:00.680363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:00.690411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:00.691812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:34:00.708960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:00.709860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:00.711252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:34:00.711252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:34:00.711252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:00.711252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:00.712144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 534/1064] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:00.716594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:00.815507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:00.819933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d8540-987f-41db-9b2d-e8942607fc2f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:00.854462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3677eaa-da12-4b80-837e-7325b75fd6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:00.855125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3677eaa-da12-4b80-837e-7325b75fd6f7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:00.855435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3677eaa-da12-4b80-837e-7325b75fd6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:00.855713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3677eaa-da12-4b80-837e-7325b75fd6f7 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:00.856426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3677eaa-da12-4b80-837e-7325b75fd6f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:00.856732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3677eaa-da12-4b80-837e-7325b75fd6f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:00.858112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3677eaa-da12-4b80-837e-7325b75fd6f7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:00.876220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 531/1065] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:00.877522 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:00.982383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f83d6-a3a1-490d-afa2-a22be89d2e75 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.000352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:01.017237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.018103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.018389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.018668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.026254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.027751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:34:01.042485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:01.043437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:01.044583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 535/1066] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:34:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:01.047487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a46811-c0cf-4a88-90c5-cd2c109558a4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.047819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a46811-c0cf-4a88-90c5-cd2c109558a4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.048103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a46811-c0cf-4a88-90c5-cd2c109558a4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.049375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.049905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 532/1067] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:01.055190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfd99e-b013-4eec-81d4-9bcfd9a07376 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.055513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfd99e-b013-4eec-81d4-9bcfd9a07376 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.055800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfd99e-b013-4eec-81d4-9bcfd9a07376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.073120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-82dfd99e-b013-4eec-81d4-9bcfd9a07376 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:01.074418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-82dfd99e-b013-4eec-81d4-9bcfd9a07376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:01.151419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 533/1068] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:34:01.164227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a698-a385-4e2a-9dde-f8c6e55ae5e7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.168448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfd99e-b013-4eec-81d4-9bcfd9a07376 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.198404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.199212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.199541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.199816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.200637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:01.200947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.202363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.219481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 536/1069] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:01.225125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.259969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:01.265334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:01.277641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfd99e-b013-4eec-81d4-9bcfd9a07376 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.320273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de598a5-7653-4fb4-a535-23097c50ca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.321458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de598a5-7653-4fb4-a535-23097c50ca56 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.321888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de598a5-7653-4fb4-a535-23097c50ca56 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.322176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de598a5-7653-4fb4-a535-23097c50ca56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.325816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8320f9e-759f-418b-9715-90c81858b21c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.333668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de598a5-7653-4fb4-a535-23097c50ca56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0de598a5-7653-4fb4-a535-23097c50ca56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:01.341454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:01.342577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:34:01.343869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 534/1070] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:34:01.350757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de598a5-7653-4fb4-a535-23097c50ca56 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.365676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.366473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.366767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.367634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.377773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.379228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:34:01.399465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:01.400435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:01.401715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:34:01.401715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:34:01.401715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:01.401715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:01.401715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 537/1071] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:01.406674 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.477740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de598a5-7653-4fb4-a535-23097c50ca56 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.506118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:01.514052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.514792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.526495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.526495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.526495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:01.526495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.526495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.526910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac71c8-82ce-438b-9b7d-39cb64c0fd94 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.532022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 535/1072] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:01.532889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.545680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:01.555974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.556644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.556952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.557199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.557906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:01.558211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.559612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.570848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 538/1073] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:01.576359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.639359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a7378-6f7f-4303-8f3e-1c24bfc9f890 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.674668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb348bd-5d40-4e46-9ee7-c2c4e5bee154 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.677034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3fa64-fa0e-4766-8fdd-f070cd05193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.677926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3fa64-fa0e-4766-8fdd-f070cd05193f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.678212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3fa64-fa0e-4766-8fdd-f070cd05193f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.678493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3fa64-fa0e-4766-8fdd-f070cd05193f None admin] PATH_INFO: `/v3/users/d398c1480db34df1973af88d2250c910` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.679763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3fa64-fa0e-4766-8fdd-f070cd05193f None admin] RBAC: Authorizing `identity:get_user(user_id=d398c1480db34df1973af88d2250c910)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.681226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3fa64-fa0e-4766-8fdd-f070cd05193f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.687679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 536/1074] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/users/d398c1480db34df1973af88d2250c910 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:01.688526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3fa64-fa0e-4766-8fdd-f070cd05193f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.715249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.716206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.716496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.716773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.723571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.725023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:34:01.739838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:01.740726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 539/1075] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:01.746762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.757977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:01.774139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:01.805223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3fa64-fa0e-4766-8fdd-f070cd05193f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.847114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bd9f2-3f4f-4a2d-a3c6-9e5790fd3708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.847996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bd9f2-3f4f-4a2d-a3c6-9e5790fd3708 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.848284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bd9f2-3f4f-4a2d-a3c6-9e5790fd3708 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.848564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bd9f2-3f4f-4a2d-a3c6-9e5790fd3708 None admin] PATH_INFO: `/v3/projects/630f17f115b441928ebf9783eb54b761` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.849732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bd9f2-3f4f-4a2d-a3c6-9e5790fd3708 None admin] RBAC: Authorizing `identity:get_project(project_id=630f17f115b441928ebf9783eb54b761)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.851708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bd9f2-3f4f-4a2d-a3c6-9e5790fd3708 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.853415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 537/1076] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/projects/630f17f115b441928ebf9783eb54b761 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:01.854566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e04f7-6c4f-4eef-b39b-d4c9816bf6d6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.854854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e04f7-6c4f-4eef-b39b-d4c9816bf6d6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.855132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e04f7-6c4f-4eef-b39b-d4c9816bf6d6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.858464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 538/1077] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:01.859229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e04f7-6c4f-4eef-b39b-d4c9816bf6d6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:01.860565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a4cff-ca06-4c3e-98ef-3e4a4df4bdbf None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:01.894369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878deb1-aea2-4d48-b334-c70083bd8b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:01.895028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878deb1-aea2-4d48-b334-c70083bd8b07 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.895342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878deb1-aea2-4d48-b334-c70083bd8b07 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.895626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878deb1-aea2-4d48-b334-c70083bd8b07 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.896401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878deb1-aea2-4d48-b334-c70083bd8b07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:01.896712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878deb1-aea2-4d48-b334-c70083bd8b07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:01.898174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878deb1-aea2-4d48-b334-c70083bd8b07 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:01.915668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 540/1078] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:34:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:01.917097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9f1c-5a70-4943-a585-d20b8cbf04cd None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:01.917394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9f1c-5a70-4943-a585-d20b8cbf04cd None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:01.917781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9f1c-5a70-4943-a585-d20b8cbf04cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:01.937595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2b9c9f1c-5a70-4943-a585-d20b8cbf04cd None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:01.938802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2b9c9f1c-5a70-4943-a585-d20b8cbf04cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:01.966530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e04f7-6c4f-4eef-b39b-d4c9816bf6d6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:02.001231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf76723-a97a-4e71-9b00-b405075c344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:02.002139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf76723-a97a-4e71-9b00-b405075c344f None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:02.002419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf76723-a97a-4e71-9b00-b405075c344f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:02.002689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf76723-a97a-4e71-9b00-b405075c344f None admin] PATH_INFO: `/v3/projects/630f17f115b441928ebf9783eb54b761/users/d398c1480db34df1973af88d2250c910/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:02.003034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:02.005132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf76723-a97a-4e71-9b00-b405075c344f None admin] RBAC: Authorizing `identity:create_grant(project_id=630f17f115b441928ebf9783eb54b761, user_id=d398c1480db34df1973af88d2250c910, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:02.006612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf76723-a97a-4e71-9b00-b405075c344f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:02.008522 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 541/1079] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:02.012124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c9f1c-5a70-4943-a585-d20b8cbf04cd None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:02.106081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c9f1c-5a70-4943-a585-d20b8cbf04cd None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:02.140402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8996480-5cb6-4897-831f-4ce07f6b08af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:02.141285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996480-5cb6-4897-831f-4ce07f6b08af None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:02.141571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996480-5cb6-4897-831f-4ce07f6b08af None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:02.141851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8996480-5cb6-4897-831f-4ce07f6b08af None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/bf670978cc2e4c529093a191bc26f885/roles/973b62e0cb57414ca5df0f3a32e38887` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:02.144115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8996480-5cb6-4897-831f-4ce07f6b08af None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=bf670978cc2e4c529093a191bc26f885, role_id=973b62e0cb57414ca5df0f3a32e38887)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:02.145597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8996480-5cb6-4897-831f-4ce07f6b08af None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:02.167785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 539/1080] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:34:01 2022] PUT /identity/v3/projects/630f17f115b441928ebf9783eb54b761/users/d398c1480db34df1973af88d2250c910/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:34:02.168609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf76723-a97a-4e71-9b00-b405075c344f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:02.177957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:02.248324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 542/1081] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:34:02 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/bf670978cc2e4c529093a191bc26f885/roles/973b62e0cb57414ca5df0f3a32e38887 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:34:02.278796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf76723-a97a-4e71-9b00-b405075c344f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:02.321379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddb2db-a3c1-4e5e-870f-858ba69f59ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:02.322071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddb2db-a3c1-4e5e-870f-858ba69f59ac None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:02.322362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddb2db-a3c1-4e5e-870f-858ba69f59ac None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:02.322644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddb2db-a3c1-4e5e-870f-858ba69f59ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:02.406979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddb2db-a3c1-4e5e-870f-858ba69f59ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:02.408522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddb2db-a3c1-4e5e-870f-858ba69f59ac None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:02.417231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:02.512410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 540/1082] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:34:02.518856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8996480-5cb6-4897-831f-4ce07f6b08af None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:02.579484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:02.622141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8996480-5cb6-4897-831f-4ce07f6b08af None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:02.658230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915874d1-d8da-46ae-b1fa-83c0b8dfb029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:02.659020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915874d1-d8da-46ae-b1fa-83c0b8dfb029 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:02.659321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915874d1-d8da-46ae-b1fa-83c0b8dfb029 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:02.659620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915874d1-d8da-46ae-b1fa-83c0b8dfb029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:02.670005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915874d1-d8da-46ae-b1fa-83c0b8dfb029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:02.671494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915874d1-d8da-46ae-b1fa-83c0b8dfb029 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-915874d1-d8da-46ae-b1fa-83c0b8dfb029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:34:02.688988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:02.690063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:02.691029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:34:02.691029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:34:02.691029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:02.691029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:02.696717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddb2db-a3c1-4e5e-870f-858ba69f59ac None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:02.699368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 543/1083] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:02.798989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddb2db-a3c1-4e5e-870f-858ba69f59ac None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:02.806475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:02.833259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede76d5d-7741-4cf6-a780-cc6610d90506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:02.833925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede76d5d-7741-4cf6-a780-cc6610d90506 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:02.834205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede76d5d-7741-4cf6-a780-cc6610d90506 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:02.834477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede76d5d-7741-4cf6-a780-cc6610d90506 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:02.835184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede76d5d-7741-4cf6-a780-cc6610d90506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:02.835512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede76d5d-7741-4cf6-a780-cc6610d90506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:02.837537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede76d5d-7741-4cf6-a780-cc6610d90506 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:02.849170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 541/1084] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:04.205671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60de8-b34c-4fad-917f-9c34a6753834 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.206232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60de8-b34c-4fad-917f-9c34a6753834 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.206232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60de8-b34c-4fad-917f-9c34a6753834 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.207627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 544/1085] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:04.213622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6442ad-5172-4aaf-b08b-3119feffc9b4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.214036 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6442ad-5172-4aaf-b08b-3119feffc9b4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.214192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6442ad-5172-4aaf-b08b-3119feffc9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.230631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-eb6442ad-5172-4aaf-b08b-3119feffc9b4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:04.231829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-eb6442ad-5172-4aaf-b08b-3119feffc9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:04.265173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18baea6-115f-45e6-bcb6-09d6f2ebdd5f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.265456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18baea6-115f-45e6-bcb6-09d6f2ebdd5f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.265730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18baea6-115f-45e6-bcb6-09d6f2ebdd5f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.267088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 545/1086] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:04.271955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454709b-5820-44df-a9d4-151af68591f6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.272242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454709b-5820-44df-a9d4-151af68591f6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.272518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454709b-5820-44df-a9d4-151af68591f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.289497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5454709b-5820-44df-a9d4-151af68591f6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:04.290704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5454709b-5820-44df-a9d4-151af68591f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:04.298379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 542/1087] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:04.321867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6442ad-5172-4aaf-b08b-3119feffc9b4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:04.357698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 546/1088] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:34:04.380908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454709b-5820-44df-a9d4-151af68591f6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:04.421485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:04.426412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:04.432613 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6442ad-5172-4aaf-b08b-3119feffc9b4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:04.465900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731fbb-9cdb-4114-94a5-cbfb6a12057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:04.466697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731fbb-9cdb-4114-94a5-cbfb6a12057c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.467039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731fbb-9cdb-4114-94a5-cbfb6a12057c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.467342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731fbb-9cdb-4114-94a5-cbfb6a12057c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.478269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8731fbb-9cdb-4114-94a5-cbfb6a12057c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:04.480944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454709b-5820-44df-a9d4-151af68591f6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-c8731fbb-9cdb-4114-94a5-cbfb6a12057c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:04.485881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:04.486993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:34:04.493180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 543/1089] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:34:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:34:04.493896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731fbb-9cdb-4114-94a5-cbfb6a12057c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:04.516271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4253682-ee91-4c30-bb28-5f3ce58ab7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:04.517083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4253682-ee91-4c30-bb28-5f3ce58ab7f7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.517371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4253682-ee91-4c30-bb28-5f3ce58ab7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.517653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4253682-ee91-4c30-bb28-5f3ce58ab7f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.528255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4253682-ee91-4c30-bb28-5f3ce58ab7f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b4253682-ee91-4c30-bb28-5f3ce58ab7f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:04.536670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:04.537601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:04.538432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 547/1090] 10.222.0.98 () {56 vars in 1120 bytes} [Fri Jul 15 15:34:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:34:04.544306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4253682-ee91-4c30-bb28-5f3ce58ab7f7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:04.594362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731fbb-9cdb-4114-94a5-cbfb6a12057c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:04.628694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:04.629377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.629665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.629946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.630691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:04.631003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:04.632438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:04.641009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 544/1091] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:34:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:04.646751 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:04.648990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:04.651991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4253682-ee91-4c30-bb28-5f3ce58ab7f7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:04.666677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:04.686201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:04.686861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.687149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.687572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.688315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:04.688626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:04.690008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:04.697969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 548/1092] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Jul 15 15:34:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:04.703817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:04.746346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093b1cb-c4f5-490d-8b33-062492f6f6cd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:04.779572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434deccb-2ba1-44c5-ac9c-502e2dac7cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:04.780552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434deccb-2ba1-44c5-ac9c-502e2dac7cf5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.780842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434deccb-2ba1-44c5-ac9c-502e2dac7cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.781126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434deccb-2ba1-44c5-ac9c-502e2dac7cf5 None admin] PATH_INFO: `/v3/users/d398c1480db34df1973af88d2250c910` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.782327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434deccb-2ba1-44c5-ac9c-502e2dac7cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=d398c1480db34df1973af88d2250c910)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:04.783817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434deccb-2ba1-44c5-ac9c-502e2dac7cf5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:04.785465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 545/1093] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:04 2022] GET /identity/v3/users/d398c1480db34df1973af88d2250c910 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:04.791149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434deccb-2ba1-44c5-ac9c-502e2dac7cf5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:04.801762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1a3f3-94a8-4a7e-a556-76be4bf807e2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:04.834505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:04.835339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.835656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.835943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.846266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:04.847873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:34:04.865151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:04.866120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:04.867764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:34:04.867764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:34:04.867764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:04.867764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:04.867764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 549/1094] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:34:04.872535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:04.874279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:04.889042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:04.890949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434deccb-2ba1-44c5-ac9c-502e2dac7cf5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:04.935896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a36fd8-ea9b-4b4a-8ede-cd9f82918299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:04.936744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36fd8-ea9b-4b4a-8ede-cd9f82918299 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:04.937096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36fd8-ea9b-4b4a-8ede-cd9f82918299 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:04.937570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36fd8-ea9b-4b4a-8ede-cd9f82918299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:04.977717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560af79-8422-45ed-9807-8ce4a7369f71 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:05.008005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a36fd8-ea9b-4b4a-8ede-cd9f82918299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.009412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a36fd8-ea9b-4b4a-8ede-cd9f82918299 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:05.011822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:05.012490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:05.012774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:05.013051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:05.013767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:05.014082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.015466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:05.027060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 550/1095] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:05.033049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:05.093571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:05.105910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:05.109260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 546/1096] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 15:34:05.115256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a36fd8-ea9b-4b4a-8ede-cd9f82918299 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:05.130172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9c09b-0619-4e6a-89cb-c7f0f6d05045 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:05.165003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:05.165788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:05.166078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:05.166359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:05.173333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.174773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:34:05.189199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:05.190208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:05.191160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 551/1097] 10.222.0.98 () {56 vars in 1093 bytes} [Fri Jul 15 15:34:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:05.196550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:05.214633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a36fd8-ea9b-4b4a-8ede-cd9f82918299 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:05.247907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925157e-869e-4cc3-929e-9eb96af6bd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:05.249097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925157e-869e-4cc3-929e-9eb96af6bd54 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:05.249097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925157e-869e-4cc3-929e-9eb96af6bd54 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:05.249304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925157e-869e-4cc3-929e-9eb96af6bd54 None admin] PATH_INFO: `/v3/projects/630f17f115b441928ebf9783eb54b761` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:05.250440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925157e-869e-4cc3-929e-9eb96af6bd54 None admin] RBAC: Authorizing `identity:get_project(project_id=630f17f115b441928ebf9783eb54b761)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.252040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925157e-869e-4cc3-929e-9eb96af6bd54 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:05.253735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 547/1098] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:05 2022] GET /identity/v3/projects/630f17f115b441928ebf9783eb54b761 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:05.260368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925157e-869e-4cc3-929e-9eb96af6bd54 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:05.296224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db120f27-b2ff-40f1-b9fc-cc5aa8ee420e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:05.314114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:05.330438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:05.331099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:05.331411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:05.331688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:05.332579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:05.332889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.334309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:05.335976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:05.352134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 552/1099] 10.222.0.98 () {56 vars in 1133 bytes} [Fri Jul 15 15:34:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:05.357928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:05.360084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925157e-869e-4cc3-929e-9eb96af6bd54 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:05.396305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:05.396964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:05.397251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:05.397531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:05.398362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=630f17f115b441928ebf9783eb54b761, user.id=d398c1480db34df1973af88d2250c910) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:05.398674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.400156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:05.409063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 548/1100] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:34:05 2022] GET /identity/v3/role_assignments?user.id=d398c1480db34df1973af88d2250c910&scope.project.id=630f17f115b441928ebf9783eb54b761&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:34:05.465023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3f91d-fe94-4d82-a0c8-c138f150fb2b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:05.500779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d5265-54cd-48ab-af91-ee4ea2165fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:05.501434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d5265-54cd-48ab-af91-ee4ea2165fe8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:05.501717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d5265-54cd-48ab-af91-ee4ea2165fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:05.501992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d5265-54cd-48ab-af91-ee4ea2165fe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:05.562861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:05.573280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d5265-54cd-48ab-af91-ee4ea2165fe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.574752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d5265-54cd-48ab-af91-ee4ea2165fe8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:05.673778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 553/1101] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 15:34:05.680652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:05.778190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f23c6f-837b-4163-8ad6-351b9d07f96b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:05.795883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:05.812772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:05.813664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:05.813943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:05.814223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:05.824568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.826105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:34:05.843666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:05.844574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:05.845488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:34:05.845488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:34:05.845488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:34:05.845488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:34:05.845488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 549/1102] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:05.850638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d5265-54cd-48ab-af91-ee4ea2165fe8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:05.941253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:05.949862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d5265-54cd-48ab-af91-ee4ea2165fe8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:05.992837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:05.993486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:05.993775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:05.994050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:05.994820 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:05.995128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:05.996526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:06.007642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 554/1103] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:06.013274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:06.110986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ceac06-df8a-4227-8853-d3014d248b67 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:06.143164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:06.143987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:06.144271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:06.144547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:06.151539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:06.152983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:34:06.167094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:06.168196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:34:06.174516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 550/1104] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:34:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:34:06.175075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:06.269168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265c9cc-175f-4ed5-8f5d-53c4869f31c3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:06.302302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729cd2a-ec02-430d-949b-22cf20e4966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:06.302953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729cd2a-ec02-430d-949b-22cf20e4966d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:06.303234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729cd2a-ec02-430d-949b-22cf20e4966d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:06.303535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729cd2a-ec02-430d-949b-22cf20e4966d None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:06.304303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729cd2a-ec02-430d-949b-22cf20e4966d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:06.304636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729cd2a-ec02-430d-949b-22cf20e4966d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:06.306051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729cd2a-ec02-430d-949b-22cf20e4966d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:06.318482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 555/1105] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:34:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:06.328899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:06.359523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:06.427756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925d43b-ef17-4ecc-b3d6-9a7c270fd49d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:06.460181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de32301-ccf5-494e-bcd7-21b1e885f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:06.460847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de32301-ccf5-494e-bcd7-21b1e885f7a6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:06.461129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de32301-ccf5-494e-bcd7-21b1e885f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:06.461405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de32301-ccf5-494e-bcd7-21b1e885f7a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:06.462221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de32301-ccf5-494e-bcd7-21b1e885f7a6 None admin] RBAC: Adding query filter params (role.id=973b62e0cb57414ca5df0f3a32e38887, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=bf670978cc2e4c529093a191bc26f885) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:06.462529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de32301-ccf5-494e-bcd7-21b1e885f7a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:06.463990 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de32301-ccf5-494e-bcd7-21b1e885f7a6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:06.469645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 551/1106] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:34:06 2022] GET /identity/v3/role_assignments?user.id=bf670978cc2e4c529093a191bc26f885&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=973b62e0cb57414ca5df0f3a32e38887 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:07.478619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35231f0d-d1ce-4468-b856-f6d4c397c78a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:07.479173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35231f0d-d1ce-4468-b856-f6d4c397c78a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:07.479173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35231f0d-d1ce-4468-b856-f6d4c397c78a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:07.480643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 556/1107] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:34:07.486218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77da50d-dc6f-4066-bcd6-a97132f4257a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:07.486645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77da50d-dc6f-4066-bcd6-a97132f4257a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:07.486782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77da50d-dc6f-4066-bcd6-a97132f4257a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:07.503203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a77da50d-dc6f-4066-bcd6-a97132f4257a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:07.504424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a77da50d-dc6f-4066-bcd6-a97132f4257a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:07.569812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 552/1108] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:07.593968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35231f0d-d1ce-4468-b856-f6d4c397c78a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:07.609116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:07.692037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35231f0d-d1ce-4468-b856-f6d4c397c78a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:07.724416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aee3b0-aba2-4417-9127-242e2dfef788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:07.725194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aee3b0-aba2-4417-9127-242e2dfef788 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:07.725477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aee3b0-aba2-4417-9127-242e2dfef788 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:07.725753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aee3b0-aba2-4417-9127-242e2dfef788 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:07.727480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aee3b0-aba2-4417-9127-242e2dfef788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:07.728912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aee3b0-aba2-4417-9127-242e2dfef788 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:07.730511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 557/1109] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:34:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:07.736354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77da50d-dc6f-4066-bcd6-a97132f4257a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:07.759764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:07.831427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77da50d-dc6f-4066-bcd6-a97132f4257a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:07.863900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d09ed5-f3a4-4c35-98ac-d0b72bf32ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:07.864550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d09ed5-f3a4-4c35-98ac-d0b72bf32ac1 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:07.864835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d09ed5-f3a4-4c35-98ac-d0b72bf32ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:07.865113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d09ed5-f3a4-4c35-98ac-d0b72bf32ac1 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:07.865813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d09ed5-f3a4-4c35-98ac-d0b72bf32ac1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:07.867279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d09ed5-f3a4-4c35-98ac-d0b72bf32ac1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:08.008240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 553/1110] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:34:07 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:34:10.062865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab73f09-698d-4f63-afec-fd0ec5e107e4 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:10.063454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab73f09-698d-4f63-afec-fd0ec5e107e4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:10.063454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab73f09-698d-4f63-afec-fd0ec5e107e4 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:10.064826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 558/1111] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:10.069945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb980-6fcb-4b4d-85b4-62a7a1513b16 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:10.070348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb980-6fcb-4b4d-85b4-62a7a1513b16 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:10.070582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcb980-6fcb-4b4d-85b4-62a7a1513b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:10.088451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5afcb980-6fcb-4b4d-85b4-62a7a1513b16 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:10.089641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5afcb980-6fcb-4b4d-85b4-62a7a1513b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:10.154103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 554/1112] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:10.178658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab73f09-698d-4f63-afec-fd0ec5e107e4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:10.257826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:10.276755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab73f09-698d-4f63-afec-fd0ec5e107e4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:10.310119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f52afa-9bca-4dcf-bf69-393731ab1dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:10.310963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52afa-9bca-4dcf-bf69-393731ab1dc9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:10.311350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52afa-9bca-4dcf-bf69-393731ab1dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:10.311636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f52afa-9bca-4dcf-bf69-393731ab1dc9 None admin] PATH_INFO: `/v3/roles/9a22f6f23fb64da488d982c17ed565ce` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:10.313147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f52afa-9bca-4dcf-bf69-393731ab1dc9 None admin] RBAC: Authorizing `identity:get_role(role_id=9a22f6f23fb64da488d982c17ed565ce)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:10.314452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f52afa-9bca-4dcf-bf69-393731ab1dc9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:10.315839 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 559/1113] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:10 2022] GET /identity/v3/roles/9a22f6f23fb64da488d982c17ed565ce => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:10.322251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcb980-6fcb-4b4d-85b4-62a7a1513b16 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:10.333695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:10.419679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcb980-6fcb-4b4d-85b4-62a7a1513b16 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:10.453216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aeaa0-339a-4ff7-8bfb-a2fc8bc0360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:10.454082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aeaa0-339a-4ff7-8bfb-a2fc8bc0360b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:10.454370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aeaa0-339a-4ff7-8bfb-a2fc8bc0360b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:10.454650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890aeaa0-339a-4ff7-8bfb-a2fc8bc0360b None admin] PATH_INFO: `/v3/users/83b0efa2af2c42bcb957586647058538` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:10.467958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890aeaa0-339a-4ff7-8bfb-a2fc8bc0360b None admin] RBAC: Authorizing `identity:get_user(user_id=83b0efa2af2c42bcb957586647058538)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:10.469424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890aeaa0-339a-4ff7-8bfb-a2fc8bc0360b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:10.471073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 555/1114] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:10 2022] GET /identity/v3/users/83b0efa2af2c42bcb957586647058538 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:10.477401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f52afa-9bca-4dcf-bf69-393731ab1dc9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:10.573081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f52afa-9bca-4dcf-bf69-393731ab1dc9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:10.606221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d716b8-2fad-46ad-b351-172b5d8057a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:10.606849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d716b8-2fad-46ad-b351-172b5d8057a7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:10.607128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d716b8-2fad-46ad-b351-172b5d8057a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:10.607429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d716b8-2fad-46ad-b351-172b5d8057a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:10.624592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:10.672173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d716b8-2fad-46ad-b351-172b5d8057a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:10.673598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d716b8-2fad-46ad-b351-172b5d8057a7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:10.768808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 560/1115] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:34:10.775449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aeaa0-339a-4ff7-8bfb-a2fc8bc0360b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:10.840349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:10.871550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890aeaa0-339a-4ff7-8bfb-a2fc8bc0360b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:10.905024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d96088-3c1a-4d66-a1ea-cbe1567bf919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:10.905873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96088-3c1a-4d66-a1ea-cbe1567bf919 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:10.906160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96088-3c1a-4d66-a1ea-cbe1567bf919 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:10.906440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d96088-3c1a-4d66-a1ea-cbe1567bf919 None admin] PATH_INFO: `/v3/projects/630f17f115b441928ebf9783eb54b761` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:10.907639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d96088-3c1a-4d66-a1ea-cbe1567bf919 None admin] RBAC: Authorizing `identity:get_project(project_id=630f17f115b441928ebf9783eb54b761)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:10.909083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d96088-3c1a-4d66-a1ea-cbe1567bf919 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:10.910756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 556/1116] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:10 2022] GET /identity/v3/projects/630f17f115b441928ebf9783eb54b761 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:10.916826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d716b8-2fad-46ad-b351-172b5d8057a7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:10.965119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:11.013435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d716b8-2fad-46ad-b351-172b5d8057a7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:11.046770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7febec-ce3b-40fb-a23a-7e934738433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:11.047649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7febec-ce3b-40fb-a23a-7e934738433a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:11.048139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7febec-ce3b-40fb-a23a-7e934738433a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:11.048617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7febec-ce3b-40fb-a23a-7e934738433a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:11.050165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7febec-ce3b-40fb-a23a-7e934738433a None admin] RBAC: Adding query filter params (role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=630f17f115b441928ebf9783eb54b761, user.id=83b0efa2af2c42bcb957586647058538) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:11.050712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7febec-ce3b-40fb-a23a-7e934738433a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:11.053399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7febec-ce3b-40fb-a23a-7e934738433a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:11.061357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 561/1117] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:34:10 2022] GET /identity/v3/role_assignments?user.id=83b0efa2af2c42bcb957586647058538&scope.project.id=630f17f115b441928ebf9783eb54b761&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:13.016922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936dd26-e469-4abd-975b-e880ad4cb0b5 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:13.017473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936dd26-e469-4abd-975b-e880ad4cb0b5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:13.017473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936dd26-e469-4abd-975b-e880ad4cb0b5 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:13.018847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 557/1118] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:13.023950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76757d59-8403-4ff8-8512-1566142a5ab7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:13.024363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76757d59-8403-4ff8-8512-1566142a5ab7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:13.024517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76757d59-8403-4ff8-8512-1566142a5ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:13.041227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-76757d59-8403-4ff8-8512-1566142a5ab7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:13.042443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-76757d59-8403-4ff8-8512-1566142a5ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:13.107837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 562/1119] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:13.130291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936dd26-e469-4abd-975b-e880ad4cb0b5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:13.228104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936dd26-e469-4abd-975b-e880ad4cb0b5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:13.260581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f7b6f-3a7b-4989-9c3c-13519c4493df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:13.261422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f7b6f-3a7b-4989-9c3c-13519c4493df None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:13.261709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f7b6f-3a7b-4989-9c3c-13519c4493df None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:13.261988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f7b6f-3a7b-4989-9c3c-13519c4493df None admin] PATH_INFO: `/v3/roles/9a22f6f23fb64da488d982c17ed565ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:13.263475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f7b6f-3a7b-4989-9c3c-13519c4493df None admin] RBAC: Authorizing `identity:get_role(role_id=9a22f6f23fb64da488d982c17ed565ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:13.264862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f7b6f-3a7b-4989-9c3c-13519c4493df None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:13.266204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 558/1120] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:13 2022] GET /identity/v3/roles/9a22f6f23fb64da488d982c17ed565ce => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:13.272007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76757d59-8403-4ff8-8512-1566142a5ab7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:13.313784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:13.367987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76757d59-8403-4ff8-8512-1566142a5ab7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:13.400470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea6618-6076-449d-9e60-d0d74cabfc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:13.401339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea6618-6076-449d-9e60-d0d74cabfc38 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:13.401624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea6618-6076-449d-9e60-d0d74cabfc38 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:13.401902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea6618-6076-449d-9e60-d0d74cabfc38 None admin] PATH_INFO: `/v3/users/83b0efa2af2c42bcb957586647058538` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:13.403070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea6618-6076-449d-9e60-d0d74cabfc38 None admin] RBAC: Authorizing `identity:get_user(user_id=83b0efa2af2c42bcb957586647058538)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:13.404536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea6618-6076-449d-9e60-d0d74cabfc38 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:13.406517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 563/1121] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:13 2022] GET /identity/v3/users/83b0efa2af2c42bcb957586647058538 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:13.412392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f7b6f-3a7b-4989-9c3c-13519c4493df None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:13.416661 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:13.510639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f7b6f-3a7b-4989-9c3c-13519c4493df None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:13.544331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92b58f-9a9f-4c28-b45c-29a8db3dcecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:13.545121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92b58f-9a9f-4c28-b45c-29a8db3dcecf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:13.545413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92b58f-9a9f-4c28-b45c-29a8db3dcecf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:13.545713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92b58f-9a9f-4c28-b45c-29a8db3dcecf None admin] PATH_INFO: `/v3/projects/630f17f115b441928ebf9783eb54b761` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:13.546927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92b58f-9a9f-4c28-b45c-29a8db3dcecf None admin] RBAC: Authorizing `identity:get_project(project_id=630f17f115b441928ebf9783eb54b761)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:13.548411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92b58f-9a9f-4c28-b45c-29a8db3dcecf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:13.550121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 559/1122] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:13 2022] GET /identity/v3/projects/630f17f115b441928ebf9783eb54b761 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:13.556254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea6618-6076-449d-9e60-d0d74cabfc38 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:13.650968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea6618-6076-449d-9e60-d0d74cabfc38 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:13.661418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:13.683836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f161d-205c-4cd1-aed0-a2752e396701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:13.684706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f161d-205c-4cd1-aed0-a2752e396701 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:13.684987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f161d-205c-4cd1-aed0-a2752e396701 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:13.685412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f161d-205c-4cd1-aed0-a2752e396701 None admin] PATH_INFO: `/v3/projects/630f17f115b441928ebf9783eb54b761/users/83b0efa2af2c42bcb957586647058538/roles/9a22f6f23fb64da488d982c17ed565ce` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:13.687803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f161d-205c-4cd1-aed0-a2752e396701 None admin] RBAC: Authorizing `identity:create_grant(project_id=630f17f115b441928ebf9783eb54b761, user_id=83b0efa2af2c42bcb957586647058538, role_id=9a22f6f23fb64da488d982c17ed565ce)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:13.689634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f161d-205c-4cd1-aed0-a2752e396701 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:13.750235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 564/1123] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:34:13 2022] PUT /identity/v3/projects/630f17f115b441928ebf9783eb54b761/users/83b0efa2af2c42bcb957586647058538/roles/9a22f6f23fb64da488d982c17ed565ce => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:34:15.730413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd460428-a48e-45ab-989a-6c9c04ad0d56 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:15.730922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd460428-a48e-45ab-989a-6c9c04ad0d56 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:15.731033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd460428-a48e-45ab-989a-6c9c04ad0d56 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:15.732402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 560/1124] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:15.737620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b1c3-fe12-4fe0-8c8c-5c567e8c2e9d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:15.738026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b1c3-fe12-4fe0-8c8c-5c567e8c2e9d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:15.738190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071b1c3-fe12-4fe0-8c8c-5c567e8c2e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:15.754627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b071b1c3-fe12-4fe0-8c8c-5c567e8c2e9d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:15.755853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b071b1c3-fe12-4fe0-8c8c-5c567e8c2e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:15.824177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 565/1125] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:15.847507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd460428-a48e-45ab-989a-6c9c04ad0d56 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:15.916595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:15.945592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd460428-a48e-45ab-989a-6c9c04ad0d56 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:15.980359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b222e7a-85da-46c6-a663-bc1dfc326d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:15.981264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222e7a-85da-46c6-a663-bc1dfc326d8e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:15.981504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222e7a-85da-46c6-a663-bc1dfc326d8e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:15.981784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b222e7a-85da-46c6-a663-bc1dfc326d8e None admin] PATH_INFO: `/v3/roles/9a22f6f23fb64da488d982c17ed565ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:15.983255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b222e7a-85da-46c6-a663-bc1dfc326d8e None admin] RBAC: Authorizing `identity:get_role(role_id=9a22f6f23fb64da488d982c17ed565ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:15.984667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b222e7a-85da-46c6-a663-bc1dfc326d8e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:15.986028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 561/1126] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:15 2022] GET /identity/v3/roles/9a22f6f23fb64da488d982c17ed565ce => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:15.992102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071b1c3-fe12-4fe0-8c8c-5c567e8c2e9d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:16.089029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:16.092050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071b1c3-fe12-4fe0-8c8c-5c567e8c2e9d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:16.125215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad3bf1-2558-48c2-a3c2-663fd9cab2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:16.126084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad3bf1-2558-48c2-a3c2-663fd9cab2a9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:16.126368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad3bf1-2558-48c2-a3c2-663fd9cab2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:16.126699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad3bf1-2558-48c2-a3c2-663fd9cab2a9 None admin] PATH_INFO: `/v3/users/83b0efa2af2c42bcb957586647058538` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:16.127907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad3bf1-2558-48c2-a3c2-663fd9cab2a9 None admin] RBAC: Authorizing `identity:get_user(user_id=83b0efa2af2c42bcb957586647058538)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:16.129391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad3bf1-2558-48c2-a3c2-663fd9cab2a9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:16.131034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 566/1127] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:15 2022] GET /identity/v3/users/83b0efa2af2c42bcb957586647058538 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:16.137451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b222e7a-85da-46c6-a663-bc1dfc326d8e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:16.232370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b222e7a-85da-46c6-a663-bc1dfc326d8e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:16.264447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3df7f-6ccf-4e0b-9fdb-bd755c8d8123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:16.265074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3df7f-6ccf-4e0b-9fdb-bd755c8d8123 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:16.265355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3df7f-6ccf-4e0b-9fdb-bd755c8d8123 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:16.265629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3df7f-6ccf-4e0b-9fdb-bd755c8d8123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:16.276149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:16.330135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3df7f-6ccf-4e0b-9fdb-bd755c8d8123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:16.331617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3df7f-6ccf-4e0b-9fdb-bd755c8d8123 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:16.432724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 562/1128] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:34:16.438809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad3bf1-2558-48c2-a3c2-663fd9cab2a9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:16.536154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad3bf1-2558-48c2-a3c2-663fd9cab2a9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:16.569400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7bb7a-14ae-4c83-8048-88e4466ad3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:16.570242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7bb7a-14ae-4c83-8048-88e4466ad3cc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:16.571087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7bb7a-14ae-4c83-8048-88e4466ad3cc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:16.571395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7bb7a-14ae-4c83-8048-88e4466ad3cc None admin] PATH_INFO: `/v3/projects/630f17f115b441928ebf9783eb54b761` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:16.572554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7bb7a-14ae-4c83-8048-88e4466ad3cc None admin] RBAC: Authorizing `identity:get_project(project_id=630f17f115b441928ebf9783eb54b761)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:16.573979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7bb7a-14ae-4c83-8048-88e4466ad3cc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:16.575659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 567/1129] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:16 2022] GET /identity/v3/projects/630f17f115b441928ebf9783eb54b761 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:16.581939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3df7f-6ccf-4e0b-9fdb-bd755c8d8123 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:16.623033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:16.678166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3df7f-6ccf-4e0b-9fdb-bd755c8d8123 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:16.711519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ebb7f-11f2-4102-b87c-45695cc98232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:16.712169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ebb7f-11f2-4102-b87c-45695cc98232 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:16.712459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ebb7f-11f2-4102-b87c-45695cc98232 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:16.712739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ebb7f-11f2-4102-b87c-45695cc98232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:16.713565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ebb7f-11f2-4102-b87c-45695cc98232 None admin] RBAC: Adding query filter params (role.id=9a22f6f23fb64da488d982c17ed565ce, scope.project.id=630f17f115b441928ebf9783eb54b761, user.id=83b0efa2af2c42bcb957586647058538) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:16.713882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ebb7f-11f2-4102-b87c-45695cc98232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:16.715353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ebb7f-11f2-4102-b87c-45695cc98232 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:16.721220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 563/1130] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:34:16 2022] GET /identity/v3/role_assignments?user.id=83b0efa2af2c42bcb957586647058538&scope.project.id=630f17f115b441928ebf9783eb54b761&role.id=9a22f6f23fb64da488d982c17ed565ce => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:18.740080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba43a-c98e-4c27-8cd2-bb8061d4b70c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:18.740604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba43a-c98e-4c27-8cd2-bb8061d4b70c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:18.740711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba43a-c98e-4c27-8cd2-bb8061d4b70c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:18.742016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 568/1131] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:18.747818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd185c-ab4f-456a-b372-a1faacf58eb5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:18.748236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd185c-ab4f-456a-b372-a1faacf58eb5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:18.748376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd185c-ab4f-456a-b372-a1faacf58eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:18.765217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-06dd185c-ab4f-456a-b372-a1faacf58eb5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:18.766387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-06dd185c-ab4f-456a-b372-a1faacf58eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:18.831322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 564/1132] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:18.853932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eba43a-c98e-4c27-8cd2-bb8061d4b70c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:18.889998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:18.953757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eba43a-c98e-4c27-8cd2-bb8061d4b70c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:18.986285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b15e-3fba-47ae-b140-db1bb4d2e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:18.987067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b15e-3fba-47ae-b140-db1bb4d2e9bd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:18.987388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b15e-3fba-47ae-b140-db1bb4d2e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:18.987671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b15e-3fba-47ae-b140-db1bb4d2e9bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:18.988827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509b15e-3fba-47ae-b140-db1bb4d2e9bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:18.990324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509b15e-3fba-47ae-b140-db1bb4d2e9bd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:18.991909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 569/1133] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:34:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:18.998477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd185c-ab4f-456a-b372-a1faacf58eb5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:19.031646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:19.101390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd185c-ab4f-456a-b372-a1faacf58eb5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:19.135744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93dbfd-79f9-40d4-b9ac-942c3c433527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:19.136404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93dbfd-79f9-40d4-b9ac-942c3c433527 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:19.136691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93dbfd-79f9-40d4-b9ac-942c3c433527 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:19.136972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93dbfd-79f9-40d4-b9ac-942c3c433527 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:19.137674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93dbfd-79f9-40d4-b9ac-942c3c433527 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:19.139118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93dbfd-79f9-40d4-b9ac-942c3c433527 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:19.249715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 565/1134] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:34:18 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:34:21.305362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc0c22-015e-42cf-ae89-c9ed0c11da79 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:21.305923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc0c22-015e-42cf-ae89-c9ed0c11da79 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:21.305923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc0c22-015e-42cf-ae89-c9ed0c11da79 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:21.307328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 570/1135] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:21.312713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebfc50-eda3-4e32-a579-8d92a2b274dd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:21.313109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebfc50-eda3-4e32-a579-8d92a2b274dd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:21.313275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebfc50-eda3-4e32-a579-8d92a2b274dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:21.330711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-64ebfc50-eda3-4e32-a579-8d92a2b274dd None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:21.331914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-64ebfc50-eda3-4e32-a579-8d92a2b274dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:21.395215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 566/1136] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:21.420524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc0c22-015e-42cf-ae89-c9ed0c11da79 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:21.518607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc0c22-015e-42cf-ae89-c9ed0c11da79 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:21.520873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:21.553420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ad6d5-294f-45d2-b986-628726325a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:21.554206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad6d5-294f-45d2-b986-628726325a75 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:21.554493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad6d5-294f-45d2-b986-628726325a75 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:21.554770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ad6d5-294f-45d2-b986-628726325a75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:21.556201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ad6d5-294f-45d2-b986-628726325a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:21.557889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ad6d5-294f-45d2-b986-628726325a75 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:21.559843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 571/1137] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:34:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:21.565670 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebfc50-eda3-4e32-a579-8d92a2b274dd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:21.583156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:21.675000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebfc50-eda3-4e32-a579-8d92a2b274dd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:21.707448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114b54-6462-44f4-997c-f16b84938f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:21.708104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114b54-6462-44f4-997c-f16b84938f93 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:21.708386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114b54-6462-44f4-997c-f16b84938f93 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:21.708660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114b54-6462-44f4-997c-f16b84938f93 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:21.709366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114b54-6462-44f4-997c-f16b84938f93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:21.710788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114b54-6462-44f4-997c-f16b84938f93 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:21.819156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 567/1138] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:34:21 2022] POST /identity/v3/users => generated 286 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:34:23.852470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579b94-4a73-4775-abf5-410f5cec666e None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:23.853020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579b94-4a73-4775-abf5-410f5cec666e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:23.853020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da579b94-4a73-4775-abf5-410f5cec666e None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:23.854387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 572/1139] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:23.859478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb418af-2943-40b6-8a23-1124d17e6a15 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:23.859858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb418af-2943-40b6-8a23-1124d17e6a15 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:23.860052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb418af-2943-40b6-8a23-1124d17e6a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:23.877108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5fb418af-2943-40b6-8a23-1124d17e6a15 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:23.878305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5fb418af-2943-40b6-8a23-1124d17e6a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:23.942749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 568/1140] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:23.965597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da579b94-4a73-4775-abf5-410f5cec666e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:24.063312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da579b94-4a73-4775-abf5-410f5cec666e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:24.095747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dad1a-89d2-4cdc-87b6-f2df12573ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:24.096520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dad1a-89d2-4cdc-87b6-f2df12573ce0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:24.096806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dad1a-89d2-4cdc-87b6-f2df12573ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:24.097085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dad1a-89d2-4cdc-87b6-f2df12573ce0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:24.106927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3dad1a-89d2-4cdc-87b6-f2df12573ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-bf3dad1a-89d2-4cdc-87b6-f2df12573ce0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:24.114371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:24.115571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:24.116611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 573/1141] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:34:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:24.122023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb418af-2943-40b6-8a23-1124d17e6a15 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:24.124369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:24.217019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb418af-2943-40b6-8a23-1124d17e6a15 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:24.249279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:24.249927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:24.250209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:24.250506 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:24.251245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:24.251580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:24.252950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:24.261409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 569/1142] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:34:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:24.267197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dad1a-89d2-4cdc-87b6-f2df12573ce0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:24.298811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:24.365619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dad1a-89d2-4cdc-87b6-f2df12573ce0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:24.398044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f36c5-5b25-4ce9-80ed-d5e80db8e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:24.398932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f36c5-5b25-4ce9-80ed-d5e80db8e086 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:24.399213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f36c5-5b25-4ce9-80ed-d5e80db8e086 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:24.399514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f36c5-5b25-4ce9-80ed-d5e80db8e086 None admin] PATH_INFO: `/v3/users/ae4beadc5a754aeba56ad8d63079df5a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:24.412617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f36c5-5b25-4ce9-80ed-d5e80db8e086 None admin] RBAC: Authorizing `identity:get_user(user_id=ae4beadc5a754aeba56ad8d63079df5a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:24.414046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f36c5-5b25-4ce9-80ed-d5e80db8e086 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:24.416231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 574/1143] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:24 2022] GET /identity/v3/users/ae4beadc5a754aeba56ad8d63079df5a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:24.421954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:24.481790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:24.518574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebaf96-61fb-4e86-8c95-2560d81fd46b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:24.551034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473f5ee-2908-4d0d-b232-70fb447a8534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:24.551694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f5ee-2908-4d0d-b232-70fb447a8534 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:24.551977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f5ee-2908-4d0d-b232-70fb447a8534 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:24.552254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473f5ee-2908-4d0d-b232-70fb447a8534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:24.616465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473f5ee-2908-4d0d-b232-70fb447a8534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:24.617920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473f5ee-2908-4d0d-b232-70fb447a8534 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:24.676667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:24.714560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 570/1144] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:34:24.720472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f36c5-5b25-4ce9-80ed-d5e80db8e086 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:24.808173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:24.818300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f36c5-5b25-4ce9-80ed-d5e80db8e086 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:24.851771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e076d-641f-4d09-9547-8cd0790d0cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:24.853225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e076d-641f-4d09-9547-8cd0790d0cde None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:24.853788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e076d-641f-4d09-9547-8cd0790d0cde None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:24.853788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e076d-641f-4d09-9547-8cd0790d0cde None admin] PATH_INFO: `/v3/projects/502680d5d3524d53b2b13e3656c21925` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:24.854934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e076d-641f-4d09-9547-8cd0790d0cde None admin] RBAC: Authorizing `identity:get_project(project_id=502680d5d3524d53b2b13e3656c21925)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:24.856398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e076d-641f-4d09-9547-8cd0790d0cde None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:24.858119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 575/1145] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:24 2022] GET /identity/v3/projects/502680d5d3524d53b2b13e3656c21925 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:24.864274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473f5ee-2908-4d0d-b232-70fb447a8534 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:24.984118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473f5ee-2908-4d0d-b232-70fb447a8534 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:25.016022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5f0a0-36de-4f6d-b43e-59be6ae5214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:25.016661 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f0a0-36de-4f6d-b43e-59be6ae5214b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:25.016945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f0a0-36de-4f6d-b43e-59be6ae5214b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:25.017221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f0a0-36de-4f6d-b43e-59be6ae5214b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:25.018032 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5f0a0-36de-4f6d-b43e-59be6ae5214b None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=502680d5d3524d53b2b13e3656c21925, user.id=ae4beadc5a754aeba56ad8d63079df5a) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:25.018342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5f0a0-36de-4f6d-b43e-59be6ae5214b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:25.019799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5f0a0-36de-4f6d-b43e-59be6ae5214b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:25.025626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 571/1146] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:34:24 2022] GET /identity/v3/role_assignments?user.id=ae4beadc5a754aeba56ad8d63079df5a&scope.project.id=502680d5d3524d53b2b13e3656c21925&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:27.001019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49600ca-382b-4256-9005-156fd667508a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:27.001543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49600ca-382b-4256-9005-156fd667508a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:27.001629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49600ca-382b-4256-9005-156fd667508a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:27.003015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 576/1147] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:27.008187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ebb41-52fd-44ba-8f36-246d3d8dfff0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:27.008547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ebb41-52fd-44ba-8f36-246d3d8dfff0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:27.008753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ebb41-52fd-44ba-8f36-246d3d8dfff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:27.025541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4a3ebb41-52fd-44ba-8f36-246d3d8dfff0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:27.026724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4a3ebb41-52fd-44ba-8f36-246d3d8dfff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:27.049343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:27.091427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 572/1148] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:27.112936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49600ca-382b-4256-9005-156fd667508a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:27.211962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49600ca-382b-4256-9005-156fd667508a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:27.244386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b15d3a-e785-4b24-9f4a-00ec8cc32aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:27.245163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b15d3a-e785-4b24-9f4a-00ec8cc32aa0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:27.245459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b15d3a-e785-4b24-9f4a-00ec8cc32aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:27.245722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b15d3a-e785-4b24-9f4a-00ec8cc32aa0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:27.255516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b15d3a-e785-4b24-9f4a-00ec8cc32aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-81b15d3a-e785-4b24-9f4a-00ec8cc32aa0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:27.263570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:27.264642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:27.265901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 577/1149] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:27.270797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ebb41-52fd-44ba-8f36-246d3d8dfff0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:27.367807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ebb41-52fd-44ba-8f36-246d3d8dfff0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:27.401098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:27.401762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:27.402056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:27.402332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:27.403073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:27.403406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:27.404796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:27.413068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 573/1150] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:27.418959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b15d3a-e785-4b24-9f4a-00ec8cc32aa0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:27.437894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:27.521711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b15d3a-e785-4b24-9f4a-00ec8cc32aa0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:27.548547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1c4a5-67d5-4853-80d2-0fdd153389b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:27.549410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1c4a5-67d5-4853-80d2-0fdd153389b6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:27.549695 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1c4a5-67d5-4853-80d2-0fdd153389b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:27.549975 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1c4a5-67d5-4853-80d2-0fdd153389b6 None admin] PATH_INFO: `/v3/users/ae4beadc5a754aeba56ad8d63079df5a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:27.551138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1c4a5-67d5-4853-80d2-0fdd153389b6 None admin] RBAC: Authorizing `identity:get_user(user_id=ae4beadc5a754aeba56ad8d63079df5a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:27.552599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1c4a5-67d5-4853-80d2-0fdd153389b6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:27.554207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 578/1151] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:27 2022] GET /identity/v3/users/ae4beadc5a754aeba56ad8d63079df5a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:27.560007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:27.582013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:27.661812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bf572-02d4-4221-893f-46284e8fb199 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:27.698497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d960a-281e-4f7e-af58-dd7931c519c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:27.699385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d960a-281e-4f7e-af58-dd7931c519c9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:27.699677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d960a-281e-4f7e-af58-dd7931c519c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:27.699953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d960a-281e-4f7e-af58-dd7931c519c9 None admin] PATH_INFO: `/v3/projects/502680d5d3524d53b2b13e3656c21925` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:27.701125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d960a-281e-4f7e-af58-dd7931c519c9 None admin] RBAC: Authorizing `identity:get_project(project_id=502680d5d3524d53b2b13e3656c21925)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:27.702695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d960a-281e-4f7e-af58-dd7931c519c9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:27.704410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 574/1152] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:27 2022] GET /identity/v3/projects/502680d5d3524d53b2b13e3656c21925 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:27.710365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1c4a5-67d5-4853-80d2-0fdd153389b6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:27.792607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:27.806969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1c4a5-67d5-4853-80d2-0fdd153389b6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:27.839252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20d211-482e-48f6-b0a2-e85b19f475c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:27.840157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20d211-482e-48f6-b0a2-e85b19f475c4 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:27.840443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20d211-482e-48f6-b0a2-e85b19f475c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:27.840722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20d211-482e-48f6-b0a2-e85b19f475c4 None admin] PATH_INFO: `/v3/projects/502680d5d3524d53b2b13e3656c21925/users/ae4beadc5a754aeba56ad8d63079df5a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:27.842656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20d211-482e-48f6-b0a2-e85b19f475c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=502680d5d3524d53b2b13e3656c21925, user_id=ae4beadc5a754aeba56ad8d63079df5a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:27.844112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20d211-482e-48f6-b0a2-e85b19f475c4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:27.916512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 579/1153] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:34:27 2022] PUT /identity/v3/projects/502680d5d3524d53b2b13e3656c21925/users/ae4beadc5a754aeba56ad8d63079df5a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:34:29.873986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e94b-a7fe-43eb-a769-8f978dc8b935 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:29.874541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e94b-a7fe-43eb-a769-8f978dc8b935 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:29.874541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5e94b-a7fe-43eb-a769-8f978dc8b935 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:29.875942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 575/1154] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:29.881332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471b0c-19f2-44ec-a48f-b65ea6463db7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:29.881722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471b0c-19f2-44ec-a48f-b65ea6463db7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:29.881895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471b0c-19f2-44ec-a48f-b65ea6463db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:29.898764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-dc471b0c-19f2-44ec-a48f-b65ea6463db7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:29.900587 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dc471b0c-19f2-44ec-a48f-b65ea6463db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:29.965201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 580/1155] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:29.988858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5e94b-a7fe-43eb-a769-8f978dc8b935 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:30.076512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:30.089056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5e94b-a7fe-43eb-a769-8f978dc8b935 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:30.121508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075bd91-c08a-4fe5-8cf5-2698e87f7e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:30.122339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075bd91-c08a-4fe5-8cf5-2698e87f7e92 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:30.122827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075bd91-c08a-4fe5-8cf5-2698e87f7e92 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:30.123330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075bd91-c08a-4fe5-8cf5-2698e87f7e92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:30.134115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075bd91-c08a-4fe5-8cf5-2698e87f7e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-3075bd91-c08a-4fe5-8cf5-2698e87f7e92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:30.141596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:30.142501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:34:30.143408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 576/1156] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:30.148767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc471b0c-19f2-44ec-a48f-b65ea6463db7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:30.223186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:30.245631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc471b0c-19f2-44ec-a48f-b65ea6463db7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:30.279089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:30.279785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:30.280074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:30.280354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:30.281375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:30.281681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:30.283046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:30.290936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 581/1157] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:34:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:30.296827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075bd91-c08a-4fe5-8cf5-2698e87f7e92 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:30.396868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075bd91-c08a-4fe5-8cf5-2698e87f7e92 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:30.429360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f736a-cb19-43db-a68d-95486bc84051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:30.430225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f736a-cb19-43db-a68d-95486bc84051 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:30.430508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f736a-cb19-43db-a68d-95486bc84051 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:30.430781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f736a-cb19-43db-a68d-95486bc84051 None admin] PATH_INFO: `/v3/users/ae4beadc5a754aeba56ad8d63079df5a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:30.431970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f736a-cb19-43db-a68d-95486bc84051 None admin] RBAC: Authorizing `identity:get_user(user_id=ae4beadc5a754aeba56ad8d63079df5a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:30.433403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f736a-cb19-43db-a68d-95486bc84051 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:30.435004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 577/1158] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:30 2022] GET /identity/v3/users/ae4beadc5a754aeba56ad8d63079df5a => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:30.440870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:30.538331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13716f3c-2c49-4791-91ea-e4b825a78707 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:30.571703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b66db5-c376-4567-a54f-899e025085af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:30.572333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66db5-c376-4567-a54f-899e025085af None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:30.572623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66db5-c376-4567-a54f-899e025085af None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:30.572905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b66db5-c376-4567-a54f-899e025085af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:30.581141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:30.638262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b66db5-c376-4567-a54f-899e025085af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:30.639734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b66db5-c376-4567-a54f-899e025085af None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:30.734912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 582/1159] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:34:30.740894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f736a-cb19-43db-a68d-95486bc84051 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:30.759915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:30.838666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f736a-cb19-43db-a68d-95486bc84051 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:30.870976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0f229-8e01-40ef-ba37-09b2e9a7c450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:30.871846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f229-8e01-40ef-ba37-09b2e9a7c450 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:30.872133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f229-8e01-40ef-ba37-09b2e9a7c450 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:30.872415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0f229-8e01-40ef-ba37-09b2e9a7c450 None admin] PATH_INFO: `/v3/projects/502680d5d3524d53b2b13e3656c21925` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:30.873573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0f229-8e01-40ef-ba37-09b2e9a7c450 None admin] RBAC: Authorizing `identity:get_project(project_id=502680d5d3524d53b2b13e3656c21925)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:30.875011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0f229-8e01-40ef-ba37-09b2e9a7c450 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:30.876708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 578/1160] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:30 2022] GET /identity/v3/projects/502680d5d3524d53b2b13e3656c21925 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:30.882763 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b66db5-c376-4567-a54f-899e025085af None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:30.921182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:30.978257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b66db5-c376-4567-a54f-899e025085af None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:31.010978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75555b-2141-4104-9573-e148699dba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:31.011591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75555b-2141-4104-9573-e148699dba83 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:31.011879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75555b-2141-4104-9573-e148699dba83 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:31.012156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75555b-2141-4104-9573-e148699dba83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:31.012973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75555b-2141-4104-9573-e148699dba83 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=502680d5d3524d53b2b13e3656c21925, user.id=ae4beadc5a754aeba56ad8d63079df5a) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:31.013285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75555b-2141-4104-9573-e148699dba83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:31.014706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75555b-2141-4104-9573-e148699dba83 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:31.020461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 583/1161] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:34:30 2022] GET /identity/v3/role_assignments?user.id=ae4beadc5a754aeba56ad8d63079df5a&scope.project.id=502680d5d3524d53b2b13e3656c21925&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:33.045038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1c34-60c6-49ef-81e4-a0e357aed2c1 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:33.045663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1c34-60c6-49ef-81e4-a0e357aed2c1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:33.045663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1c34-60c6-49ef-81e4-a0e357aed2c1 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:33.046960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 579/1162] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:33.052070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d86ff2-b2e2-4ccb-83f4-d05367d1be89 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:33.052508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d86ff2-b2e2-4ccb-83f4-d05367d1be89 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:33.052636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d86ff2-b2e2-4ccb-83f4-d05367d1be89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:33.068952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-21d86ff2-b2e2-4ccb-83f4-d05367d1be89 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:33.070119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-21d86ff2-b2e2-4ccb-83f4-d05367d1be89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:33.133622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 584/1163] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:33.155511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1c34-60c6-49ef-81e4-a0e357aed2c1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:33.238352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:33.253994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda1c34-60c6-49ef-81e4-a0e357aed2c1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:33.287345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb19dd-b79a-480f-887e-549c24961fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:33.287989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb19dd-b79a-480f-887e-549c24961fbc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:33.288276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb19dd-b79a-480f-887e-549c24961fbc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:33.288553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb19dd-b79a-480f-887e-549c24961fbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:33.353161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb19dd-b79a-480f-887e-549c24961fbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:33.354550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb19dd-b79a-480f-887e-549c24961fbc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:33.434221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:33.450504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 580/1164] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 15:34:33.456507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d86ff2-b2e2-4ccb-83f4-d05367d1be89 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:33.486009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:33.552321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d86ff2-b2e2-4ccb-83f4-d05367d1be89 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:33.585099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ff9ec-8e40-4ecb-a7e5-05c4c9b9ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:33.585882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ff9ec-8e40-4ecb-a7e5-05c4c9b9ce8b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:33.586167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ff9ec-8e40-4ecb-a7e5-05c4c9b9ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:33.586463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ff9ec-8e40-4ecb-a7e5-05c4c9b9ce8b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:33.593546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ff9ec-8e40-4ecb-a7e5-05c4c9b9ce8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:33.594990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ff9ec-8e40-4ecb-a7e5-05c4c9b9ce8b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a37ff9ec-8e40-4ecb-a7e5-05c4c9b9ce8b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:33.609090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:33.610337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:33.611527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 585/1165] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:34:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:33.616352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb19dd-b79a-480f-887e-549c24961fbc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:33.716510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb19dd-b79a-480f-887e-549c24961fbc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:33.752144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420dc13-b3b6-4d32-a53a-17987a704c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:33.752805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420dc13-b3b6-4d32-a53a-17987a704c9f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:33.753095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420dc13-b3b6-4d32-a53a-17987a704c9f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:33.753376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420dc13-b3b6-4d32-a53a-17987a704c9f None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:33.754097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420dc13-b3b6-4d32-a53a-17987a704c9f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:33.754406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420dc13-b3b6-4d32-a53a-17987a704c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:33.755820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420dc13-b3b6-4d32-a53a-17987a704c9f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:33.763577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 581/1166] 10.222.0.98 () {56 vars in 1115 bytes} [Fri Jul 15 15:34:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:35.766446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f23a6-1b2e-4fe7-869a-069d77f0d6cc None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:35.766974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f23a6-1b2e-4fe7-869a-069d77f0d6cc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:35.767096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f23a6-1b2e-4fe7-869a-069d77f0d6cc None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:35.768408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 586/1167] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:35.773510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9bbb-e28f-4e93-9153-1bf4091e8a83 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:35.773959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9bbb-e28f-4e93-9153-1bf4091e8a83 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:35.774165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9bbb-e28f-4e93-9153-1bf4091e8a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:35.797485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6a9b9bbb-e28f-4e93-9153-1bf4091e8a83 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:35.799236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6a9b9bbb-e28f-4e93-9153-1bf4091e8a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:35.847022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:35.870344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 582/1168] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:35.893343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f23a6-1b2e-4fe7-869a-069d77f0d6cc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:35.984760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:35.994098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f23a6-1b2e-4fe7-869a-069d77f0d6cc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:36.027742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2a0a1-02f2-411c-b5ca-31542746729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:36.028336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a0a1-02f2-411c-b5ca-31542746729d None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:36.028623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a0a1-02f2-411c-b5ca-31542746729d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:36.028903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2a0a1-02f2-411c-b5ca-31542746729d None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:36.029643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2a0a1-02f2-411c-b5ca-31542746729d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:36.031004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2a0a1-02f2-411c-b5ca-31542746729d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:36.083431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 587/1169] 10.222.0.98 () {60 vars in 1138 bytes} [Fri Jul 15 15:34:35 2022] POST /identity/v3/domains => generated 267 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:34:38.105196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8541d-e5cb-451d-89ba-528236997b15 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:38.105754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8541d-e5cb-451d-89ba-528236997b15 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:38.105754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8541d-e5cb-451d-89ba-528236997b15 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:38.107152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 583/1170] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:38.112402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb80cbc-c98d-4519-a55d-4d07f0e5d7ae None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:38.112809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb80cbc-c98d-4519-a55d-4d07f0e5d7ae None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:38.112961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb80cbc-c98d-4519-a55d-4d07f0e5d7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:38.129256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fbb80cbc-c98d-4519-a55d-4d07f0e5d7ae None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:38.130437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fbb80cbc-c98d-4519-a55d-4d07f0e5d7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:38.199550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 588/1171] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:38.222292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8541d-e5cb-451d-89ba-528236997b15 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:38.321718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8541d-e5cb-451d-89ba-528236997b15 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:38.354143 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0b263-55f6-4203-8479-abd23c5875d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:38.354996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0b263-55f6-4203-8479-abd23c5875d0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:38.355362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0b263-55f6-4203-8479-abd23c5875d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:38.355595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0b263-55f6-4203-8479-abd23c5875d0 None admin] PATH_INFO: `/v3/domains/8bee006303b34ab5a481ecc4920645a4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:38.365986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0b263-55f6-4203-8479-abd23c5875d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8bee006303b34ab5a481ecc4920645a4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:38.367482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0b263-55f6-4203-8479-abd23c5875d0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:38.369833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 584/1172] 10.222.0.98 () {56 vars in 1183 bytes} [Fri Jul 15 15:34:38 2022] GET /identity/v3/domains/8bee006303b34ab5a481ecc4920645a4 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:38.376050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb80cbc-c98d-4519-a55d-4d07f0e5d7ae None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:38.456170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:38.473287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb80cbc-c98d-4519-a55d-4d07f0e5d7ae None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:38.506705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeffecc-ba0b-4911-8873-82a6f3cb91bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:38.507341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffecc-ba0b-4911-8873-82a6f3cb91bb None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:38.508383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffecc-ba0b-4911-8873-82a6f3cb91bb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:38.508383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffecc-ba0b-4911-8873-82a6f3cb91bb None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:38.509178 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeffecc-ba0b-4911-8873-82a6f3cb91bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:38.510656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeffecc-ba0b-4911-8873-82a6f3cb91bb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:38.569205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 589/1173] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:34:38 2022] POST /identity/v3/projects => generated 361 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:34:40.563093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ef503-a7bf-47b1-887e-a5dd0b48dcb6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:40.563713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ef503-a7bf-47b1-887e-a5dd0b48dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:40.563713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ef503-a7bf-47b1-887e-a5dd0b48dcb6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:40.565094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 585/1174] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:40.570342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48656e-129e-465e-9ab6-6fb450a5aa3a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:40.570756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48656e-129e-465e-9ab6-6fb450a5aa3a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:40.570880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48656e-129e-465e-9ab6-6fb450a5aa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:40.587896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6e48656e-129e-465e-9ab6-6fb450a5aa3a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:40.589070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6e48656e-129e-465e-9ab6-6fb450a5aa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:40.652604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 590/1175] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:34:40.677014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ef503-a7bf-47b1-887e-a5dd0b48dcb6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:40.719688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:40.785241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ef503-a7bf-47b1-887e-a5dd0b48dcb6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:40.829439 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b21032-58f0-424b-8a0c-58197a1e70d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:40.830920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b21032-58f0-424b-8a0c-58197a1e70d8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:40.831443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b21032-58f0-424b-8a0c-58197a1e70d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:40.831937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b21032-58f0-424b-8a0c-58197a1e70d8 None admin] PATH_INFO: `/v3/domains/8bee006303b34ab5a481ecc4920645a4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:40.833732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b21032-58f0-424b-8a0c-58197a1e70d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8bee006303b34ab5a481ecc4920645a4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:40.835427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b21032-58f0-424b-8a0c-58197a1e70d8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:40.837183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 586/1176] 10.222.0.98 () {56 vars in 1183 bytes} [Fri Jul 15 15:34:40 2022] GET /identity/v3/domains/8bee006303b34ab5a481ecc4920645a4 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:40.843042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48656e-129e-465e-9ab6-6fb450a5aa3a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:40.912918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:40.951823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48656e-129e-465e-9ab6-6fb450a5aa3a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:40.989089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92eb509-0949-44e7-8528-86a79c77ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:40.989748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eb509-0949-44e7-8528-86a79c77ab93 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:40.990034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eb509-0949-44e7-8528-86a79c77ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:40.990312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eb509-0949-44e7-8528-86a79c77ab93 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:40.991004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92eb509-0949-44e7-8528-86a79c77ab93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:40.992462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92eb509-0949-44e7-8528-86a79c77ab93 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:41.065981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 591/1177] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:34:40 2022] POST /identity/v3/users => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:34:43.161863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f142821-3dd2-4fd7-b112-cf157ddc2bf1 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:43.162414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f142821-3dd2-4fd7-b112-cf157ddc2bf1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:43.162414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f142821-3dd2-4fd7-b112-cf157ddc2bf1 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:43.163944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 587/1178] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:43.169717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadd27f-29fd-4702-9ba4-a69029da4e81 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:43.170144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadd27f-29fd-4702-9ba4-a69029da4e81 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:43.170283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadd27f-29fd-4702-9ba4-a69029da4e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:43.187802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7eadd27f-29fd-4702-9ba4-a69029da4e81 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:43.188999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7eadd27f-29fd-4702-9ba4-a69029da4e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:43.253027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 592/1179] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:43.276361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f142821-3dd2-4fd7-b112-cf157ddc2bf1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:43.375717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f142821-3dd2-4fd7-b112-cf157ddc2bf1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:43.382409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:43.409089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce9c4-00a8-4093-9ad9-cbd7d6ab7408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:43.410109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce9c4-00a8-4093-9ad9-cbd7d6ab7408 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:43.410394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce9c4-00a8-4093-9ad9-cbd7d6ab7408 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:43.410671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce9c4-00a8-4093-9ad9-cbd7d6ab7408 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:43.421147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ce9c4-00a8-4093-9ad9-cbd7d6ab7408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-fb7ce9c4-00a8-4093-9ad9-cbd7d6ab7408 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:43.428566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:43.429667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:34:43.431467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 588/1180] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:34:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:43.436426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadd27f-29fd-4702-9ba4-a69029da4e81 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:43.484867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:43.534703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadd27f-29fd-4702-9ba4-a69029da4e81 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:43.567691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:43.568356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:43.568645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:43.568924 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:43.569667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:43.569977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:43.571378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:43.579867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 593/1181] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:34:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:43.585705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce9c4-00a8-4093-9ad9-cbd7d6ab7408 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:43.689587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce9c4-00a8-4093-9ad9-cbd7d6ab7408 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:43.721842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a5f5d-6d30-4fb4-b9bb-ab97e10e84e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:43.722706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5f5d-6d30-4fb4-b9bb-ab97e10e84e9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:43.722992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5f5d-6d30-4fb4-b9bb-ab97e10e84e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:43.723270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a5f5d-6d30-4fb4-b9bb-ab97e10e84e9 None admin] PATH_INFO: `/v3/users/cb1505578d5d47239de9da1ad489604a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:43.736508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a5f5d-6d30-4fb4-b9bb-ab97e10e84e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cb1505578d5d47239de9da1ad489604a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:43.737958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a5f5d-6d30-4fb4-b9bb-ab97e10e84e9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:43.739630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 589/1182] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:43 2022] GET /identity/v3/users/cb1505578d5d47239de9da1ad489604a => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:43.745435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:43.848300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd6c36-e9b6-4964-893a-61e0a65f1d54 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:43.860503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:43.880483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941ce2-3146-416c-92bf-5a561d2028ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:43.897416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941ce2-3146-416c-92bf-5a561d2028ff None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:43.897416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941ce2-3146-416c-92bf-5a561d2028ff None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:43.897416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941ce2-3146-416c-92bf-5a561d2028ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:43.949092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941ce2-3146-416c-92bf-5a561d2028ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:43.950482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941ce2-3146-416c-92bf-5a561d2028ff None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:44.079724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 594/1183] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:34:44.085654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a5f5d-6d30-4fb4-b9bb-ab97e10e84e9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:44.111098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:44.182015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a5f5d-6d30-4fb4-b9bb-ab97e10e84e9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:44.215741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024f3ca-c2e0-4aa1-9c0d-ec6e26659eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:44.216588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024f3ca-c2e0-4aa1-9c0d-ec6e26659eb9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:44.216874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024f3ca-c2e0-4aa1-9c0d-ec6e26659eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:44.217152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024f3ca-c2e0-4aa1-9c0d-ec6e26659eb9 None admin] PATH_INFO: `/v3/projects/a7c1c93fb832424d87d4737c4685e038` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:44.218319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024f3ca-c2e0-4aa1-9c0d-ec6e26659eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=a7c1c93fb832424d87d4737c4685e038)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:44.219875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024f3ca-c2e0-4aa1-9c0d-ec6e26659eb9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:44.221558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 590/1184] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:44 2022] GET /identity/v3/projects/a7c1c93fb832424d87d4737c4685e038 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:44.227640 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941ce2-3146-416c-92bf-5a561d2028ff None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:44.242050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:44.323897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941ce2-3146-416c-92bf-5a561d2028ff None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:44.356994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fcbee-321a-441d-9a9a-2d6046be962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:44.357634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fcbee-321a-441d-9a9a-2d6046be962c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:44.357925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fcbee-321a-441d-9a9a-2d6046be962c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:44.358205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fcbee-321a-441d-9a9a-2d6046be962c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:44.359023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fcbee-321a-441d-9a9a-2d6046be962c None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=a7c1c93fb832424d87d4737c4685e038, user.id=cb1505578d5d47239de9da1ad489604a) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:44.359355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fcbee-321a-441d-9a9a-2d6046be962c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:44.360782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fcbee-321a-441d-9a9a-2d6046be962c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:44.366055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 595/1185] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:34:44 2022] GET /identity/v3/role_assignments?user.id=cb1505578d5d47239de9da1ad489604a&scope.project.id=a7c1c93fb832424d87d4737c4685e038&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:46.335579 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41595f4b-3bc9-4669-a04e-e94d6c8ad72a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:46.336123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41595f4b-3bc9-4669-a04e-e94d6c8ad72a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:46.336123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41595f4b-3bc9-4669-a04e-e94d6c8ad72a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:46.337520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 591/1186] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:46.342830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a09381-df78-40a1-8a52-751f1ed43878 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:46.343227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a09381-df78-40a1-8a52-751f1ed43878 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:46.343430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a09381-df78-40a1-8a52-751f1ed43878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:46.361130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-96a09381-df78-40a1-8a52-751f1ed43878 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:46.362317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-96a09381-df78-40a1-8a52-751f1ed43878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:46.427082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 596/1187] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:46.449210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41595f4b-3bc9-4669-a04e-e94d6c8ad72a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:46.538578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:46.548318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41595f4b-3bc9-4669-a04e-e94d6c8ad72a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:46.580423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa5b0f-0cba-4dd9-a0b7-39486f2eabde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:46.581204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5b0f-0cba-4dd9-a0b7-39486f2eabde None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:46.581494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5b0f-0cba-4dd9-a0b7-39486f2eabde None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:46.581774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5b0f-0cba-4dd9-a0b7-39486f2eabde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:46.592736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa5b0f-0cba-4dd9-a0b7-39486f2eabde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b6aa5b0f-0cba-4dd9-a0b7-39486f2eabde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:46.600231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:46.601318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:34:46.602360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 592/1188] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:34:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:46.608274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a09381-df78-40a1-8a52-751f1ed43878 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:46.616405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:46.704173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a09381-df78-40a1-8a52-751f1ed43878 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:46.737549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:46.738197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:46.738480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:46.738756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:46.739512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:46.739817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:46.741176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:46.749035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 597/1189] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:34:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:46.754953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa5b0f-0cba-4dd9-a0b7-39486f2eabde None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:46.854694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa5b0f-0cba-4dd9-a0b7-39486f2eabde None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:46.887941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277840a5-bc2f-4723-9ac5-6faf59624a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:46.888805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277840a5-bc2f-4723-9ac5-6faf59624a6c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:46.889090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277840a5-bc2f-4723-9ac5-6faf59624a6c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:46.889371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277840a5-bc2f-4723-9ac5-6faf59624a6c None admin] PATH_INFO: `/v3/users/cb1505578d5d47239de9da1ad489604a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:46.890681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277840a5-bc2f-4723-9ac5-6faf59624a6c None admin] RBAC: Authorizing `identity:get_user(user_id=cb1505578d5d47239de9da1ad489604a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:46.892158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277840a5-bc2f-4723-9ac5-6faf59624a6c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:46.893777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 593/1190] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:46 2022] GET /identity/v3/users/cb1505578d5d47239de9da1ad489604a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:46.899589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:46.964570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:46.996132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53816947-2f64-402f-80ed-b2dc68f79167 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:47.031013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437745f5-dcb8-42df-8496-8524d02c4394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:47.032151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437745f5-dcb8-42df-8496-8524d02c4394 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:47.032440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437745f5-dcb8-42df-8496-8524d02c4394 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:47.032715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437745f5-dcb8-42df-8496-8524d02c4394 None admin] PATH_INFO: `/v3/projects/a7c1c93fb832424d87d4737c4685e038` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:47.033881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437745f5-dcb8-42df-8496-8524d02c4394 None admin] RBAC: Authorizing `identity:get_project(project_id=a7c1c93fb832424d87d4737c4685e038)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:47.035331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437745f5-dcb8-42df-8496-8524d02c4394 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:47.037019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 598/1191] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:46 2022] GET /identity/v3/projects/a7c1c93fb832424d87d4737c4685e038 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:47.043140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277840a5-bc2f-4723-9ac5-6faf59624a6c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:47.064832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:47.140853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277840a5-bc2f-4723-9ac5-6faf59624a6c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:47.173435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a435b3-f266-433e-8b7c-a19af1b6a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:47.174322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a435b3-f266-433e-8b7c-a19af1b6a20a None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:47.174619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a435b3-f266-433e-8b7c-a19af1b6a20a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:47.174900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a435b3-f266-433e-8b7c-a19af1b6a20a None admin] PATH_INFO: `/v3/projects/a7c1c93fb832424d87d4737c4685e038/users/cb1505578d5d47239de9da1ad489604a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:47.176895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a435b3-f266-433e-8b7c-a19af1b6a20a None admin] RBAC: Authorizing `identity:create_grant(project_id=a7c1c93fb832424d87d4737c4685e038, user_id=cb1505578d5d47239de9da1ad489604a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:47.178339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a435b3-f266-433e-8b7c-a19af1b6a20a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:47.278042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 594/1192] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:34:47 2022] PUT /identity/v3/projects/a7c1c93fb832424d87d4737c4685e038/users/cb1505578d5d47239de9da1ad489604a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:34:49.235550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069adda-f464-4c02-b159-2a2a8002f160 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:49.236102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069adda-f464-4c02-b159-2a2a8002f160 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:49.236102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069adda-f464-4c02-b159-2a2a8002f160 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:49.237466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 599/1193] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:34:49.242764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85675-ad9f-479f-abd2-500a6b0b198a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:49.243203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85675-ad9f-479f-abd2-500a6b0b198a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:49.243361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde85675-ad9f-479f-abd2-500a6b0b198a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:49.260271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-bde85675-ad9f-479f-abd2-500a6b0b198a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:49.261449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bde85675-ad9f-479f-abd2-500a6b0b198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:49.326142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 595/1194] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:34:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:34:49.348911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069adda-f464-4c02-b159-2a2a8002f160 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:49.451916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069adda-f464-4c02-b159-2a2a8002f160 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:49.484193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145b01-5dc5-4321-a9c1-98ff6a0945f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:49.484969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145b01-5dc5-4321-a9c1-98ff6a0945f6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:49.485256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145b01-5dc5-4321-a9c1-98ff6a0945f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:49.485613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a145b01-5dc5-4321-a9c1-98ff6a0945f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:49.488174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:49.496956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a145b01-5dc5-4321-a9c1-98ff6a0945f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0a145b01-5dc5-4321-a9c1-98ff6a0945f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:34:49.504417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:34:49.505622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:34:49.506672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 600/1195] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:34:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:34:49.512433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde85675-ad9f-479f-abd2-500a6b0b198a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:49.524122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:49.608580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde85675-ad9f-479f-abd2-500a6b0b198a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:49.642137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:49.642790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:49.643078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:49.643383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:49.644125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:49.644435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:49.645810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:49.654628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 596/1196] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:34:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:49.660747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145b01-5dc5-4321-a9c1-98ff6a0945f6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:49.756897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a145b01-5dc5-4321-a9c1-98ff6a0945f6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:49.790526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba04836-e1ab-4d10-86e2-08d7e1594f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:49.791407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba04836-e1ab-4d10-86e2-08d7e1594f22 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:49.791694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba04836-e1ab-4d10-86e2-08d7e1594f22 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:49.791970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba04836-e1ab-4d10-86e2-08d7e1594f22 None admin] PATH_INFO: `/v3/users/cb1505578d5d47239de9da1ad489604a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:49.793204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba04836-e1ab-4d10-86e2-08d7e1594f22 None admin] RBAC: Authorizing `identity:get_user(user_id=cb1505578d5d47239de9da1ad489604a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:49.794653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba04836-e1ab-4d10-86e2-08d7e1594f22 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:49.796282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 601/1197] 10.222.0.98 () {56 vars in 1177 bytes} [Fri Jul 15 15:34:49 2022] GET /identity/v3/users/cb1505578d5d47239de9da1ad489604a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:49.801829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:49.870161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:49.904242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eecef-dd40-4a97-91b2-9a5357bd5408 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:49.936763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081212-8458-4d1b-a8a4-155d4cb6c212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:49.937410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081212-8458-4d1b-a8a4-155d4cb6c212 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:49.937696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081212-8458-4d1b-a8a4-155d4cb6c212 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:49.937974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081212-8458-4d1b-a8a4-155d4cb6c212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:50.007330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081212-8458-4d1b-a8a4-155d4cb6c212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:50.008753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081212-8458-4d1b-a8a4-155d4cb6c212 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:50.078598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:50.107534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 597/1198] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:34:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:34:50.113433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba04836-e1ab-4d10-86e2-08d7e1594f22 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:50.173802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:50.210176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba04836-e1ab-4d10-86e2-08d7e1594f22 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:50.244171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247ab9f-26ec-4ecc-8cfb-b4d5280ea7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:50.245013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247ab9f-26ec-4ecc-8cfb-b4d5280ea7a3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:50.245295 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247ab9f-26ec-4ecc-8cfb-b4d5280ea7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:50.245571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247ab9f-26ec-4ecc-8cfb-b4d5280ea7a3 None admin] PATH_INFO: `/v3/projects/a7c1c93fb832424d87d4737c4685e038` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:50.246729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247ab9f-26ec-4ecc-8cfb-b4d5280ea7a3 None admin] RBAC: Authorizing `identity:get_project(project_id=a7c1c93fb832424d87d4737c4685e038)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:50.248180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247ab9f-26ec-4ecc-8cfb-b4d5280ea7a3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:50.249939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 602/1199] 10.222.0.98 () {56 vars in 1186 bytes} [Fri Jul 15 15:34:50 2022] GET /identity/v3/projects/a7c1c93fb832424d87d4737c4685e038 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:34:50.256018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081212-8458-4d1b-a8a4-155d4cb6c212 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:50.354701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081212-8458-4d1b-a8a4-155d4cb6c212 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:50.386966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aac39e-246e-4410-aea9-3e24db9a46c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:50.387651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aac39e-246e-4410-aea9-3e24db9a46c3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:50.387940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aac39e-246e-4410-aea9-3e24db9a46c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:50.388220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aac39e-246e-4410-aea9-3e24db9a46c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:50.389037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aac39e-246e-4410-aea9-3e24db9a46c3 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=a7c1c93fb832424d87d4737c4685e038, user.id=cb1505578d5d47239de9da1ad489604a) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:34:50.389350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aac39e-246e-4410-aea9-3e24db9a46c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:50.390787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aac39e-246e-4410-aea9-3e24db9a46c3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:50.396524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 598/1200] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:34:50 2022] GET /identity/v3/role_assignments?user.id=cb1505578d5d47239de9da1ad489604a&scope.project.id=a7c1c93fb832424d87d4737c4685e038&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:35:00.240077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=34 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:00.393900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=82 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:07.554689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ec646-e021-4bbc-9b1a-416e7a087a22 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:07.555266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ec646-e021-4bbc-9b1a-416e7a087a22 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:07.555266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ec646-e021-4bbc-9b1a-416e7a087a22 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:07.556670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 603/1201] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:07.561676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c07b1d-b615-49eb-bf7e-2ce47fc64309 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:07.562135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c07b1d-b615-49eb-bf7e-2ce47fc64309 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:07.562249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c07b1d-b615-49eb-bf7e-2ce47fc64309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:07.580028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-55c07b1d-b615-49eb-bf7e-2ce47fc64309 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:07.581518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-55c07b1d-b615-49eb-bf7e-2ce47fc64309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:07.649835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 599/1202] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:07.674135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ec646-e021-4bbc-9b1a-416e7a087a22 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:07.774303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ec646-e021-4bbc-9b1a-416e7a087a22 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:07.808533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:07.809311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:07.809592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:07.809869 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:07.819563 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:07.820991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:07.837619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:07.838769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:07.839960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:07.839960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:07.839960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:07.839960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:07.839960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 604/1203] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:07.844912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c07b1d-b615-49eb-bf7e-2ce47fc64309 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:07.943346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c07b1d-b615-49eb-bf7e-2ce47fc64309 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:07.970531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:07.976689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b0fe7-5e4f-40b0-bb87-44e7692c0f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:07.977359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0fe7-5e4f-40b0-bb87-44e7692c0f3c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:07.977656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0fe7-5e4f-40b0-bb87-44e7692c0f3c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:07.977935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b0fe7-5e4f-40b0-bb87-44e7692c0f3c None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:07.978652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b0fe7-5e4f-40b0-bb87-44e7692c0f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:07.978960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b0fe7-5e4f-40b0-bb87-44e7692c0f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:07.980368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b0fe7-5e4f-40b0-bb87-44e7692c0f3c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:07.991825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 600/1204] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:07.997733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:08.050830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:08.095803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12a490-5219-4483-9316-fce82043dbad None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:08.128601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e94ca-7220-4fef-9ab4-ad47e9a9cc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:08.129369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e94ca-7220-4fef-9ab4-ad47e9a9cc20 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:08.129658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e94ca-7220-4fef-9ab4-ad47e9a9cc20 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:08.129936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e94ca-7220-4fef-9ab4-ad47e9a9cc20 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:08.130777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e94ca-7220-4fef-9ab4-ad47e9a9cc20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:08.132302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e94ca-7220-4fef-9ab4-ad47e9a9cc20 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:08.384650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 605/1205] 10.222.0.98 () {60 vars in 1132 bytes} [Fri Jul 15 15:36:07 2022] POST /identity/v3/users => generated 251 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:36:10.428525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd5f3-e832-4e8d-a9e7-76b3790439d0 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:10.429080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd5f3-e832-4e8d-a9e7-76b3790439d0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:10.429080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd5f3-e832-4e8d-a9e7-76b3790439d0 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:10.430481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 601/1206] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:10.435823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43de2a-97f8-42f6-b287-c9cb515dc088 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:10.436247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43de2a-97f8-42f6-b287-c9cb515dc088 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:10.436379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43de2a-97f8-42f6-b287-c9cb515dc088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:10.453132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2d43de2a-97f8-42f6-b287-c9cb515dc088 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:10.454335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2d43de2a-97f8-42f6-b287-c9cb515dc088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:10.520390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 606/1207] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:10.544431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd5f3-e832-4e8d-a9e7-76b3790439d0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:10.644708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd5f3-e832-4e8d-a9e7-76b3790439d0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:10.678480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f6bad-bfc6-464f-b7ed-aa8d5a3874ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:10.679209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f6bad-bfc6-464f-b7ed-aa8d5a3874ea None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:10.679527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f6bad-bfc6-464f-b7ed-aa8d5a3874ea None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:10.679805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f6bad-bfc6-464f-b7ed-aa8d5a3874ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:10.691463 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f6bad-bfc6-464f-b7ed-aa8d5a3874ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-f42f6bad-bfc6-464f-b7ed-aa8d5a3874ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:36:10.699115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:10.700213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:10.701385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 602/1208] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:36:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:10.707059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43de2a-97f8-42f6-b287-c9cb515dc088 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:10.746314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:10.803890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43de2a-97f8-42f6-b287-c9cb515dc088 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:10.837431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:10.838093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:10.838376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:10.838655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:10.839485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:10.839796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:10.841175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:10.848865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 607/1209] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:36:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:10.854620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f6bad-bfc6-464f-b7ed-aa8d5a3874ea None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:10.904140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:10.953733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f6bad-bfc6-464f-b7ed-aa8d5a3874ea None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:10.986665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:10.987510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:10.987799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:10.988081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:10.997939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:10.999434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:11.017780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:11.019431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:11.020656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:11.020656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:11.020656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:11.020656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:11.020656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 603/1210] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:11.025427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:11.124175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:11.127114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16888b4-db80-4f73-852d-cdb1812808db None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:11.160069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:11.160777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:11.161057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:11.161329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:11.162060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:11.162360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:11.164059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:11.177348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 608/1211] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:11.182992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:11.279914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f486da0-6568-4335-a27a-924fc110f3ca None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:11.281520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:11.312910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:11.313703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:11.313991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:11.314272 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:11.321604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:11.323049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:36:11.337465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:11.338565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:11.339852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 604/1212] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:11.344496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:11.445506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1623ab-63e1-4ca1-91fc-8e5d1beafa32 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:11.478824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:11.479526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:11.479817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:11.480098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:11.480917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:11.481232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:11.482658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:11.499256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 609/1213] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Jul 15 15:36:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:11.505688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:11.604428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc127ff6-d9c2-433d-ad29-df910cc382f5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:11.637431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7490e84-b415-45c6-b874-386d7742b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:11.640921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7490e84-b415-45c6-b874-386d7742b22d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:11.640921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7490e84-b415-45c6-b874-386d7742b22d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:11.640921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7490e84-b415-45c6-b874-386d7742b22d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:11.668803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:11.705377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7490e84-b415-45c6-b874-386d7742b22d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:11.707563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7490e84-b415-45c6-b874-386d7742b22d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:11.804349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 605/1214] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:36:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:36:11.810352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:11.829797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:11.906720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a62511-fbed-402b-8c42-36c17607c2ba None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:11.946035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:11.946832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:11.947118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:11.947424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:11.957536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:11.958989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:11.976276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:11.977331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:11.983247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:11.983247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:11.983247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:11.983247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:11.983247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 610/1215] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:11.983733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7490e84-b415-45c6-b874-386d7742b22d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:12.044312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:12.082266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7490e84-b415-45c6-b874-386d7742b22d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:12.115340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:12.116220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:12.116512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:12.116792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:12.117520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:12.117819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:12.119200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:12.131831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 606/1216] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:12.137750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:12.206438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:12.236052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b213684-3277-44b8-8a6c-aaf4961c63ae None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:12.268644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:12.269476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:12.269759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:12.270036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:12.277091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:12.278519 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:36:12.293313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:12.294390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:12.297672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 611/1217] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:36:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:12.301127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:12.399529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e3bda-a2be-4a07-99b2-cdae8f580411 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:12.422263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:12.434175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ace00f-4800-4491-80e3-f3ce3677d45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:12.434843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace00f-4800-4491-80e3-f3ce3677d45e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:12.435134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace00f-4800-4491-80e3-f3ce3677d45e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:12.435447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace00f-4800-4491-80e3-f3ce3677d45e None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:12.436232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ace00f-4800-4491-80e3-f3ce3677d45e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:12.436546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ace00f-4800-4491-80e3-f3ce3677d45e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:12.437993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ace00f-4800-4491-80e3-f3ce3677d45e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:12.449739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 607/1218] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:36:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:36:12.455894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:12.555641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753f161-699d-4cc3-b3b8-e918e1857c04 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:12.583078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:12.588629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc725e-1c62-43e5-8497-59573ff7d8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:12.589276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc725e-1c62-43e5-8497-59573ff7d8bb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:12.589561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc725e-1c62-43e5-8497-59573ff7d8bb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:12.589851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc725e-1c62-43e5-8497-59573ff7d8bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:12.590649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc725e-1c62-43e5-8497-59573ff7d8bb None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=e57e2b49a8624266b12c78a78059cc80) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:12.590958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc725e-1c62-43e5-8497-59573ff7d8bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:12.592417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc725e-1c62-43e5-8497-59573ff7d8bb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:12.598286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 612/1219] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:36:12 2022] GET /identity/v3/role_assignments?user.id=e57e2b49a8624266b12c78a78059cc80&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:14.626400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6ac67-4171-4942-829c-682888c1129f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:14.626935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6ac67-4171-4942-829c-682888c1129f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:14.627042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6ac67-4171-4942-829c-682888c1129f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:14.628483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 608/1220] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:14.633744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aed1b-436b-4877-96cb-b31c75a05486 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:14.634193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aed1b-436b-4877-96cb-b31c75a05486 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:14.634319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aed1b-436b-4877-96cb-b31c75a05486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:14.650888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8e5aed1b-436b-4877-96cb-b31c75a05486 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:14.652118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8e5aed1b-436b-4877-96cb-b31c75a05486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:14.718897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 613/1221] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:14.742526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6ac67-4171-4942-829c-682888c1129f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:14.841995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6ac67-4171-4942-829c-682888c1129f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:14.875105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206f14e-e811-4a73-b07c-9ad40a422bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:14.875922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206f14e-e811-4a73-b07c-9ad40a422bd4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:14.876208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206f14e-e811-4a73-b07c-9ad40a422bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:14.876481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206f14e-e811-4a73-b07c-9ad40a422bd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:14.886265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206f14e-e811-4a73-b07c-9ad40a422bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0206f14e-e811-4a73-b07c-9ad40a422bd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:36:14.894570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:14.895724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:14.909942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 609/1222] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:36:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:14.910852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5aed1b-436b-4877-96cb-b31c75a05486 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:14.999529 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5aed1b-436b-4877-96cb-b31c75a05486 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.023028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:15.033640 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:15.034894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.035177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.035483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.036217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:15.036517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:15.037880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:15.049174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 614/1223] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:36:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:15.051887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206f14e-e811-4a73-b07c-9ad40a422bd4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:15.094855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:15.147829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206f14e-e811-4a73-b07c-9ad40a422bd4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.180664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:15.181469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.181760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.182043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.193276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:15.194741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:15.212440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:15.213526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:15.214627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:15.214627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:15.214627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:15.214627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:15.214627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 610/1224] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:15.221073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:15.319198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a4e69-8697-4e0a-b433-c44243f5d3d7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.351630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:15.352284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.352570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.352850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.353561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:15.353874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:15.355242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:15.366682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 615/1225] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:15.372536 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:15.467085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:15.473739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4e7c2-6fea-47ae-836a-6d9e45bba6f1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.508263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:15.509067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.509352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.509626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.516654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:15.518109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:36:15.533607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:15.540973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:15.542139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 611/1226] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:15.542893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:15.587625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:15.638724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865370a0-c7b1-4d4c-ac85-c8724ac28a76 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.671889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:15.672549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.672836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.673112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.673952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:15.674260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:15.676258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:15.693353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 616/1227] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Jul 15 15:36:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:36:15.699146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:15.795597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84716e-a4e3-40fc-912e-fceffc08de9e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.832843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:15.833648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.833934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.834214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.844350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:15.845761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:15.863633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:15.864682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:15.865617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:15.865617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:15.865617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:15.865617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:15.865617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 612/1228] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:15.870741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:15.969234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c76cd7-8275-477b-9022-67c1cc327069 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.971497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:16.001931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:16.002571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:16.003384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:16.003663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:16.004385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:16.004673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:16.006078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:16.017224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 617/1229] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:16.023356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:16.032220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:16.121275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6db7b-f4b4-4df6-9377-2a34708f451b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:16.155406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:16.156288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:16.156578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:16.156860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:16.164650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:16.169517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:36:16.180355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:16.181325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:16.182225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 613/1230] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:36:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:36:16.187539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:16.287486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030bda4-4b01-4510-ab07-aba21d08fd20 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:16.321774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26344b2-be2b-4652-b155-ceccf4bbd437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:16.322427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26344b2-be2b-4652-b155-ceccf4bbd437 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:16.322711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26344b2-be2b-4652-b155-ceccf4bbd437 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:16.322986 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26344b2-be2b-4652-b155-ceccf4bbd437 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:16.323790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26344b2-be2b-4652-b155-ceccf4bbd437 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:16.324097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26344b2-be2b-4652-b155-ceccf4bbd437 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:16.325577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26344b2-be2b-4652-b155-ceccf4bbd437 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:16.337480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 618/1231] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:36:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:16.343736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:16.409457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:16.444926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec699889-b692-4dd3-8c58-9a29b4a0e565 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:16.477961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a1e04-714b-4089-98b2-aa532fc69682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:16.478882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1e04-714b-4089-98b2-aa532fc69682 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:16.479163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1e04-714b-4089-98b2-aa532fc69682 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:16.479467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1e04-714b-4089-98b2-aa532fc69682 None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/e57e2b49a8624266b12c78a78059cc80/roles/75570e166e6e4bd680698565f2d2cc0b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:16.496620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a1e04-714b-4089-98b2-aa532fc69682 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=e57e2b49a8624266b12c78a78059cc80, role_id=75570e166e6e4bd680698565f2d2cc0b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:16.498160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a1e04-714b-4089-98b2-aa532fc69682 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:16.614322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 614/1232] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:36:16 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/e57e2b49a8624266b12c78a78059cc80/roles/75570e166e6e4bd680698565f2d2cc0b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:36:18.620313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708ba03-9cc5-4944-8c37-177d1627da57 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:18.620858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708ba03-9cc5-4944-8c37-177d1627da57 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:18.620858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708ba03-9cc5-4944-8c37-177d1627da57 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:18.622250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 619/1233] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:18.627443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0551369-a481-48b7-bd91-a13f3df05240 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:18.627844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0551369-a481-48b7-bd91-a13f3df05240 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:18.628663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0551369-a481-48b7-bd91-a13f3df05240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:18.644784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e0551369-a481-48b7-bd91-a13f3df05240 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:18.645974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e0551369-a481-48b7-bd91-a13f3df05240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:18.713578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 615/1234] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:18.736741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708ba03-9cc5-4944-8c37-177d1627da57 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:18.762286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:18.836938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708ba03-9cc5-4944-8c37-177d1627da57 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:18.870434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6dfd-b936-469f-9664-2419a6b714fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:18.871213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6dfd-b936-469f-9664-2419a6b714fb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:18.871523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6dfd-b936-469f-9664-2419a6b714fb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:18.871802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6dfd-b936-469f-9664-2419a6b714fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:18.882327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab6dfd-b936-469f-9664-2419a6b714fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-87ab6dfd-b936-469f-9664-2419a6b714fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:36:18.890110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:18.891231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:18.893927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 620/1235] 10.222.0.98 () {56 vars in 1102 bytes} [Fri Jul 15 15:36:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:18.897852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0551369-a481-48b7-bd91-a13f3df05240 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:18.943588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:18.995097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0551369-a481-48b7-bd91-a13f3df05240 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.026810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.027500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.028163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.028163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.028767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:19.029069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.030448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.039539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 616/1236] 10.222.0.98 () {56 vars in 1103 bytes} [Fri Jul 15 15:36:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:19.045237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6dfd-b936-469f-9664-2419a6b714fb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.130847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:19.146438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6dfd-b936-469f-9664-2419a6b714fb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.179896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.180728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.181017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.181294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.191981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.193447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:19.210289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:19.211518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:19.212670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:19.212670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:19.212670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:19.212670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:19.212670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 621/1237] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:19.217648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.314654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa03df2-0487-4e5e-84e4-0f800972ba77 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.323610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:19.347757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.348414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.348700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.348976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.349689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:19.349991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.351441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.362400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 617/1238] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:19.368159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.465637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444adb4c-bec5-431a-9515-e997ebe5228a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.498545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.499365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.499649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.499940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] PATH_INFO: `/v3/users/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.502889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:19.508088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.509518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:36:19.525899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:19.526993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:19.527994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 622/1239] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:19.532995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.633489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92a1fe-8a43-429e-8a18-1efcbe5693fb None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.666879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.667571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.667862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.668146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.668974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:19.669289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.670740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.687606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 618/1240] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:19.694200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.793930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a3591-fdd3-4bd5-9683-4fff93a3ea4a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.827512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee52db-8cc1-43d3-a242-4ccb9bba7e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.828155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee52db-8cc1-43d3-a242-4ccb9bba7e16 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.828444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee52db-8cc1-43d3-a242-4ccb9bba7e16 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.828727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee52db-8cc1-43d3-a242-4ccb9bba7e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.888403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:19.895176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee52db-8cc1-43d3-a242-4ccb9bba7e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.896668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee52db-8cc1-43d3-a242-4ccb9bba7e16 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.990896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 623/1241] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:36:19.996754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:20.024824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:20.096039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368aa18-e0e8-4f48-a797-c32df21fa031 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:20.129957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:20.130744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:20.131030 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:20.131318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:20.141254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:20.142668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:20.159708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:20.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:20.161852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:20.161852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:20.161852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:20.161852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:20.161852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 619/1242] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:20.166938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee52db-8cc1-43d3-a242-4ccb9bba7e16 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:20.259986 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:20.265608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee52db-8cc1-43d3-a242-4ccb9bba7e16 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:20.298162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:20.298815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:20.299103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:20.299412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:20.300732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:20.301044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:20.302409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:20.313697 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 624/1243] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:20.320030 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:20.396664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:20.417739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2171fd-6b3f-4e16-99ec-8d214d17fee9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:20.451649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:20.452477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:20.452767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:20.453054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:20.459631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:20.461085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:36:20.475522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:20.476621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:20.477884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 620/1244] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:36:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:20.482636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:20.580487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec58b49-d683-4b14-b367-cbd003430fec None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:20.615041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59131335-6c22-4871-8bb7-034bb1d0cbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:20.615724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131335-6c22-4871-8bb7-034bb1d0cbda None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:20.616009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131335-6c22-4871-8bb7-034bb1d0cbda None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:20.616285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59131335-6c22-4871-8bb7-034bb1d0cbda None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:20.617054 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59131335-6c22-4871-8bb7-034bb1d0cbda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:20.617363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59131335-6c22-4871-8bb7-034bb1d0cbda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:20.618827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59131335-6c22-4871-8bb7-034bb1d0cbda None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:20.630569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 625/1245] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:36:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:20.636787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:20.738231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac6ac9-137f-4135-92dd-169699bbcf24 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:20.770261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76f1bc-514d-4da4-a61e-6cde0edf9d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:20.770902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f1bc-514d-4da4-a61e-6cde0edf9d55 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:20.771188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f1bc-514d-4da4-a61e-6cde0edf9d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:20.771498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f1bc-514d-4da4-a61e-6cde0edf9d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:20.772329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76f1bc-514d-4da4-a61e-6cde0edf9d55 None admin] RBAC: Adding query filter params (role.id=75570e166e6e4bd680698565f2d2cc0b, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=e57e2b49a8624266b12c78a78059cc80) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:20.772644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76f1bc-514d-4da4-a61e-6cde0edf9d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:20.774101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76f1bc-514d-4da4-a61e-6cde0edf9d55 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:20.780091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 621/1246] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:36:20 2022] GET /identity/v3/role_assignments?user.id=e57e2b49a8624266b12c78a78059cc80&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=75570e166e6e4bd680698565f2d2cc0b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:22.832047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422e6d-2c0e-4f84-9073-7c87195d5135 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:22.832600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422e6d-2c0e-4f84-9073-7c87195d5135 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:22.832600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422e6d-2c0e-4f84-9073-7c87195d5135 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:22.834012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 626/1247] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:22.839139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724954-cf33-409a-80f9-99bb7c0685f9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:22.839583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724954-cf33-409a-80f9-99bb7c0685f9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:22.839746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13724954-cf33-409a-80f9-99bb7c0685f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:22.844789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:22.856729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-13724954-cf33-409a-80f9-99bb7c0685f9 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:22.857923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-13724954-cf33-409a-80f9-99bb7c0685f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:22.923849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 622/1248] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:22.947632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43422e6d-2c0e-4f84-9073-7c87195d5135 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:22.966064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:23.046425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43422e6d-2c0e-4f84-9073-7c87195d5135 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:23.078893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfcc7f-2498-4449-a32b-22fde5dad786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:23.079706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfcc7f-2498-4449-a32b-22fde5dad786 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:23.079992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfcc7f-2498-4449-a32b-22fde5dad786 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:23.080273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfcc7f-2498-4449-a32b-22fde5dad786 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:23.090695 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfcc7f-2498-4449-a32b-22fde5dad786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-5bbfcc7f-2498-4449-a32b-22fde5dad786 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:36:23.098770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:23.099908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:23.100934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 627/1249] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:36:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:23.106065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13724954-cf33-409a-80f9-99bb7c0685f9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:23.207164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13724954-cf33-409a-80f9-99bb7c0685f9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:23.236076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:23.240907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:23.241612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:23.241896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:23.242169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:23.242900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:23.243203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:23.244608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:23.252499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 623/1250] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:23.258413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfcc7f-2498-4449-a32b-22fde5dad786 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:23.334696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:23.356354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfcc7f-2498-4449-a32b-22fde5dad786 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:23.388163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:23.393675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:23.393675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:23.393675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:23.401074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:23.402450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:23.419818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:23.426748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:23.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:23.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:23.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:23.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:23.427981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 628/1251] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:23.428333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:23.523718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a8ac5-6379-4fd5-b283-772b07301678 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:23.555860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:23.556515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:23.556796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:23.557069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:23.557778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:23.558083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:23.559484 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:23.571568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 624/1252] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:23.577248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:23.677592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfac3b-81bb-41a7-994e-24aa267788c2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:23.703117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:23.711346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:23.712136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:23.712419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:23.712714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] PATH_INFO: `/v3/users/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:23.719911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:23.721333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:36:23.737402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:23.739809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:23.740745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 629/1253] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:23.744544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:23.801039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:23.846932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72950074-d4e8-49d3-a0fa-e54a373645d1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:23.881560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:23.882224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:23.882510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:23.882785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:23.883663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:23.883970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:23.885406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:23.902268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 625/1254] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Jul 15 15:36:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:23.908122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.008304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0f672-a63d-40e1-8a90-2e9fb671472d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.040732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47a50a-1ec1-4b70-ad6d-f2d37adecaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.041373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47a50a-1ec1-4b70-ad6d-f2d37adecaa4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.041659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47a50a-1ec1-4b70-ad6d-f2d37adecaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.041938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47a50a-1ec1-4b70-ad6d-f2d37adecaa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.095894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:24.107777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47a50a-1ec1-4b70-ad6d-f2d37adecaa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.109229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47a50a-1ec1-4b70-ad6d-f2d37adecaa4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.206346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 630/1255] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:36:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:36:24.212396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.311716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c4778-f706-4a7b-b6e5-497957966861 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.324450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:24.344366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.345156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.345444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.345722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.356000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.357455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:24.375067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:24.376220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:24.377318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:24.377318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:24.377318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:24.377318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:24.377318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 626/1256] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:24.382261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47a50a-1ec1-4b70-ad6d-f2d37adecaa4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.467838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:24.479646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47a50a-1ec1-4b70-ad6d-f2d37adecaa4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.511848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.512497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.512780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.513055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.513758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:24.514062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.515464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.526213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 631/1257] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:24.532086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.630964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac952f4-3749-40c2-93b0-5e9788fee294 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.665353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.666176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.666457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.666742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.673410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.674890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:36:24.688342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:24.689320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:24.690224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 627/1258] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:36:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:24.695619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.795370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d2712-a482-4c35-bc1d-d98de45994ff None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.828461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f3783-85a7-4b2a-bdc6-bc75e48d6b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.829237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f3783-85a7-4b2a-bdc6-bc75e48d6b1b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.829568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f3783-85a7-4b2a-bdc6-bc75e48d6b1b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.829850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f3783-85a7-4b2a-bdc6-bc75e48d6b1b None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.830669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f3783-85a7-4b2a-bdc6-bc75e48d6b1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:24.831037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f3783-85a7-4b2a-bdc6-bc75e48d6b1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.833201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f3783-85a7-4b2a-bdc6-bc75e48d6b1b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.845368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 632/1259] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:36:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:24.852036 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.877189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:24.949785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504ce3-ef5c-4fbf-acae-982c892fcdd7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.983519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bcf22-3174-41ce-a8ae-5be255356de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.984177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bcf22-3174-41ce-a8ae-5be255356de8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.984468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bcf22-3174-41ce-a8ae-5be255356de8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.984750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bcf22-3174-41ce-a8ae-5be255356de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.985577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7bcf22-3174-41ce-a8ae-5be255356de8 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=e57e2b49a8624266b12c78a78059cc80) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:24.985897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7bcf22-3174-41ce-a8ae-5be255356de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.987366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7bcf22-3174-41ce-a8ae-5be255356de8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.992737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 628/1260] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:36:24 2022] GET /identity/v3/role_assignments?user.id=e57e2b49a8624266b12c78a78059cc80&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:27.013129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e903de-959e-4276-bf13-7243ce84880b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:27.013731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e903de-959e-4276-bf13-7243ce84880b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:27.013731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e903de-959e-4276-bf13-7243ce84880b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:27.015099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 633/1261] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:27.020418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e596c-9697-4220-91e0-5cf95c38c184 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:27.020805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e596c-9697-4220-91e0-5cf95c38c184 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:27.020992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e596c-9697-4220-91e0-5cf95c38c184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:27.037967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-645e596c-9697-4220-91e0-5cf95c38c184 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:27.039465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-645e596c-9697-4220-91e0-5cf95c38c184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:27.105718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 629/1262] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:27.128953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e903de-959e-4276-bf13-7243ce84880b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:27.139934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:27.230667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e903de-959e-4276-bf13-7243ce84880b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:27.264123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3530d00-a4ac-48a0-afec-56474f31d459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:27.264845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3530d00-a4ac-48a0-afec-56474f31d459 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:27.265130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3530d00-a4ac-48a0-afec-56474f31d459 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:27.265410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3530d00-a4ac-48a0-afec-56474f31d459 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:27.275870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3530d00-a4ac-48a0-afec-56474f31d459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f3530d00-a4ac-48a0-afec-56474f31d459 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:36:27.283635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:27.285362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:27.286362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 634/1263] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:36:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:27.292040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e596c-9697-4220-91e0-5cf95c38c184 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:27.310868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:27.389130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e596c-9697-4220-91e0-5cf95c38c184 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:27.422594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:27.424520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:27.424520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:27.424520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:27.424860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:27.424950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:27.426962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:27.435104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 630/1264] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:36:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:27.441096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3530d00-a4ac-48a0-afec-56474f31d459 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:27.507134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:27.540131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3530d00-a4ac-48a0-afec-56474f31d459 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:27.573066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:27.574848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:27.574848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:27.574848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:27.585275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:27.586780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:27.604953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:27.606074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:27.607095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:27.607095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:27.607095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:27.607095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:27.607095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 635/1265] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:27.612097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:27.691729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:27.709943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8c39-f16b-4c22-942a-dfec059d0f7f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:27.741678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:27.742335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:27.742619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:27.742889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:27.743634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:27.743937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:27.745318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:27.756626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 631/1266] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:27.762434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:27.861663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f3cee-3773-41ce-a1f6-0f478475f7c7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:27.880647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:27.894702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:27.895584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:27.895867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:27.896137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:27.903538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:27.904968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:36:27.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:27.925234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:27.926190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 636/1267] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:27.926802 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:28.023439 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee8cf2-16cf-4d8f-8f1c-f36b65f941db None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:28.056524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:28.063658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:28.063658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:28.063658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:28.063658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:28.063658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:28.063658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:28.069549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:28.078185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 632/1268] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Jul 15 15:36:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:28.084695 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:28.183016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081ca35-8235-4060-81d6-7353d9104cc5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:28.217369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:28.218149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:28.218527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:28.218808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:28.230017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:28.231567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:28.248381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:28.249439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:28.250351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:28.250351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:28.250351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:28.250351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:28.250351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 637/1269] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:28.256007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:28.353168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37521c-1b5d-456e-aeda-8e7cbd3f71e5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:28.387207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:28.387922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:28.388213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:28.388495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:28.389218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:28.389529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:28.390950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:28.402278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 633/1270] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:28.408036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:28.440056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:28.507019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149ba7e-bc80-4c43-ba8c-262e2952e23c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:28.544309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:28.546654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:28.546654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:28.546654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:28.557421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:28.558883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:36:28.573662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:28.574872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:28.575895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 638/1271] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:36:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:28.582471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:28.641740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:28.683004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83967098-3d29-4c05-82bb-d3d767ddd934 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:28.715482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2defc6-089a-43cb-b733-347d857f072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:28.716130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2defc6-089a-43cb-b733-347d857f072b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:28.716414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2defc6-089a-43cb-b733-347d857f072b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:28.716690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2defc6-089a-43cb-b733-347d857f072b None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:28.717459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2defc6-089a-43cb-b733-347d857f072b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:28.717763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2defc6-089a-43cb-b733-347d857f072b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:28.719200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2defc6-089a-43cb-b733-347d857f072b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:28.731355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 634/1272] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:36:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:28.737530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:28.829551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:28.837642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc005064-5bdd-4b97-bdb9-0c792cca5f0c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:28.872302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11fd25-e032-4c22-b748-38ad766d305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:28.873209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11fd25-e032-4c22-b748-38ad766d305b None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:28.873490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11fd25-e032-4c22-b748-38ad766d305b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:28.873764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11fd25-e032-4c22-b748-38ad766d305b None admin] PATH_INFO: `/v3/projects/7ffc55a6490647cfa8f543334e168150/users/e57e2b49a8624266b12c78a78059cc80/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:28.876308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11fd25-e032-4c22-b748-38ad766d305b None admin] RBAC: Authorizing `identity:create_grant(project_id=7ffc55a6490647cfa8f543334e168150, user_id=e57e2b49a8624266b12c78a78059cc80, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:28.877847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11fd25-e032-4c22-b748-38ad766d305b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:28.996304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 639/1273] 10.222.0.98 () {58 vars in 1439 bytes} [Fri Jul 15 15:36:28 2022] PUT /identity/v3/projects/7ffc55a6490647cfa8f543334e168150/users/e57e2b49a8624266b12c78a78059cc80/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:36:30.970323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865fdcb-4d40-4a4a-99d4-177ef3dde2e4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:30.970877 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865fdcb-4d40-4a4a-99d4-177ef3dde2e4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:30.970877 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865fdcb-4d40-4a4a-99d4-177ef3dde2e4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:30.972825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 635/1274] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:30.978019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1e712-b984-4728-8edc-4d39dc9f1336 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:30.978422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1e712-b984-4728-8edc-4d39dc9f1336 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:30.978580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1e712-b984-4728-8edc-4d39dc9f1336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:30.995247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c5c1e712-b984-4728-8edc-4d39dc9f1336 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:30.996518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c5c1e712-b984-4728-8edc-4d39dc9f1336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:31.061226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 640/1275] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:31.084208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865fdcb-4d40-4a4a-99d4-177ef3dde2e4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:31.188705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865fdcb-4d40-4a4a-99d4-177ef3dde2e4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:31.205319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:31.230322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efda41-76b5-4217-950a-240940cf768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:31.231658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efda41-76b5-4217-950a-240940cf768a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:31.232176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efda41-76b5-4217-950a-240940cf768a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:31.232679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efda41-76b5-4217-950a-240940cf768a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:31.247496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efda41-76b5-4217-950a-240940cf768a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-54efda41-76b5-4217-950a-240940cf768a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:36:31.256450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:31.257641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:31.263018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 636/1276] 10.222.0.98 () {56 vars in 1096 bytes} [Fri Jul 15 15:36:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:31.264096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1e712-b984-4728-8edc-4d39dc9f1336 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:31.353236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:31.370547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1e712-b984-4728-8edc-4d39dc9f1336 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:31.403162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:31.403856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:31.404139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:31.404413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:31.405139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:31.405500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:31.407163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:31.415591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 641/1277] 10.222.0.98 () {56 vars in 1099 bytes} [Fri Jul 15 15:36:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:31.421281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efda41-76b5-4217-950a-240940cf768a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:31.520331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efda41-76b5-4217-950a-240940cf768a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:31.554604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:31.555546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:31.555884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:31.556198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:31.566990 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:31.568494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:31.586008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:31.588334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:31.589486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:31.589486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:31.589486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:31.589486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:31.589486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 637/1278] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:31.592974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:31.696131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b897e-bd16-4fef-8104-971093ac0feb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:31.728484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:31.729146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:31.729434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:31.729714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:31.730428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:31.730733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:31.732132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:31.745014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 642/1279] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:36:31.750792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:31.781227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:31.849273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a374cab-bd22-4ed2-ab71-d2b1d86f12e0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:31.881280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:31.882119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:31.882611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:31.883125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:31.891485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:31.892943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:36:31.907067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:31.908228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:31.909417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 638/1280] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:31.915016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:31.918793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:32.013163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3270b1-7669-4f6a-9d76-eb4591b49054 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:32.046515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:32.047140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:32.047459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:32.047742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:32.048559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:32.048868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:32.050349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:32.067646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 643/1281] 10.222.0.98 () {56 vars in 1143 bytes} [Fri Jul 15 15:36:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:32.073417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:32.163997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:32.173867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f212f-a6e4-4f0a-bcea-5c7395e829ff None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:32.206012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56743-8f62-40ca-8216-4c9e70b45dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:32.207166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56743-8f62-40ca-8216-4c9e70b45dc8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:32.207734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56743-8f62-40ca-8216-4c9e70b45dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:32.208015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56743-8f62-40ca-8216-4c9e70b45dc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:32.271881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56743-8f62-40ca-8216-4c9e70b45dc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:32.273339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56743-8f62-40ca-8216-4c9e70b45dc8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:32.377436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:32.377436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 639/1282] 10.222.0.98 () {58 vars in 1282 bytes} [Fri Jul 15 15:36:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:36:32.377719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:32.435407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:32.473878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d880e0-0888-47c2-b14a-4f43f53706e2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:32.506817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:32.507628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:32.507917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:32.508192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:32.517821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:32.519838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:36:32.536780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:32.547372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:32.548496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:36:32.548496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:36:32.548496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:36:32.548496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:32.548496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 644/1283] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:32.548832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56743-8f62-40ca-8216-4c9e70b45dc8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:32.641482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56743-8f62-40ca-8216-4c9e70b45dc8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:32.675970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:32.676619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:32.676899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:32.677172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:32.677880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:32.678181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:32.679565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:32.690599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 640/1284] 10.222.0.98 () {56 vars in 1109 bytes} [Fri Jul 15 15:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:32.696711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:32.794631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc829664-4512-4cdb-9490-4212c9aa56c4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:32.809019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:32.829168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:32.829946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:32.830228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:32.830503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:32.837272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:32.839050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:36:32.853289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:32.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jul 15 15:36:32.855276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 645/1285] 10.222.0.98 () {56 vars in 1111 bytes} [Fri Jul 15 15:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:32.860714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:32.910630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:32.960888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3af70-11e8-4b19-9892-a83b49e9ba0a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:32.993405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469147da-59ad-4e6b-aec3-33a40f3878ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:32.994063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469147da-59ad-4e6b-aec3-33a40f3878ee None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:32.994349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469147da-59ad-4e6b-aec3-33a40f3878ee None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:32.994627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469147da-59ad-4e6b-aec3-33a40f3878ee None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:32.995437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469147da-59ad-4e6b-aec3-33a40f3878ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:32.995748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469147da-59ad-4e6b-aec3-33a40f3878ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:32.997328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469147da-59ad-4e6b-aec3-33a40f3878ee None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:33.010453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 641/1286] 10.222.0.98 () {56 vars in 1148 bytes} [Fri Jul 15 15:36:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:33.016731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:33.115862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1406e57-1d4e-4700-a3b6-ce1f2e1bbd05 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:33.149056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec574ee4-e488-4113-8841-5dab9d1e3fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:33.149694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec574ee4-e488-4113-8841-5dab9d1e3fb5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:33.149982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec574ee4-e488-4113-8841-5dab9d1e3fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:33.154438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec574ee4-e488-4113-8841-5dab9d1e3fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:33.154438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec574ee4-e488-4113-8841-5dab9d1e3fb5 None admin] RBAC: Adding query filter params (role.id=9813667366e34ffeba8eccc9db7c90eb, scope.project.id=7ffc55a6490647cfa8f543334e168150, user.id=e57e2b49a8624266b12c78a78059cc80) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:33.154438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec574ee4-e488-4113-8841-5dab9d1e3fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:33.154438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec574ee4-e488-4113-8841-5dab9d1e3fb5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:33.158557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 646/1287] 10.222.0.98 () {56 vars in 1374 bytes} [Fri Jul 15 15:36:33 2022] GET /identity/v3/role_assignments?user.id=e57e2b49a8624266b12c78a78059cc80&scope.project.id=7ffc55a6490647cfa8f543334e168150&role.id=9813667366e34ffeba8eccc9db7c90eb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:35.164292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528ef3-b8fa-4fb5-9fa2-efda30024c31 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:35.165093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528ef3-b8fa-4fb5-9fa2-efda30024c31 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:35.165198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528ef3-b8fa-4fb5-9fa2-efda30024c31 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:35.166516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 642/1288] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:35.171834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ba206-1fc5-44c8-9420-156a8112261a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:35.172279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ba206-1fc5-44c8-9420-156a8112261a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:35.172376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ba206-1fc5-44c8-9420-156a8112261a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:35.189371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8e6ba206-1fc5-44c8-9420-156a8112261a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:35.190551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8e6ba206-1fc5-44c8-9420-156a8112261a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:35.219280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:35.254818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 647/1289] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:35.281345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528ef3-b8fa-4fb5-9fa2-efda30024c31 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:35.384057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528ef3-b8fa-4fb5-9fa2-efda30024c31 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:35.395790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:35.419370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:35.420132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:35.420418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:35.420700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] PATH_INFO: `/v3/services/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:35.421351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:35.422741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:35.434764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:35.443595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:35.444638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 643/1290] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:36:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:35.445505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ba206-1fc5-44c8-9420-156a8112261a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:35.540912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ba206-1fc5-44c8-9420-156a8112261a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:35.574089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197c608-ac3f-4aae-9464-15400d971f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:35.574740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197c608-ac3f-4aae-9464-15400d971f69 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:35.575025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197c608-ac3f-4aae-9464-15400d971f69 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:35.575318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197c608-ac3f-4aae-9464-15400d971f69 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:35.576020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197c608-ac3f-4aae-9464-15400d971f69 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:35.576327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197c608-ac3f-4aae-9464-15400d971f69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:35.577713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197c608-ac3f-4aae-9464-15400d971f69 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:35.583169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 648/1291] 10.222.0.98 () {56 vars in 1116 bytes} [Fri Jul 15 15:36:35 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:36:35.589004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:35.687509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b7154-2f02-411c-8d6d-b251bb1205da None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:35.722093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934794b6-71c6-40f1-b306-a165c57df1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:35.722780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934794b6-71c6-40f1-b306-a165c57df1eb None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:35.723451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934794b6-71c6-40f1-b306-a165c57df1eb None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:35.723451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934794b6-71c6-40f1-b306-a165c57df1eb None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:35.724198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934794b6-71c6-40f1-b306-a165c57df1eb None admin] RBAC: Adding query filter params (type=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:35.724509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934794b6-71c6-40f1-b306-a165c57df1eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:35.725888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934794b6-71c6-40f1-b306-a165c57df1eb None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:35.730300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 644/1292] 10.222.0.98 () {56 vars in 1116 bytes} [Fri Jul 15 15:36:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:37.721877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a062-fd59-4ded-9364-1acd27821b9b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:37.722387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a062-fd59-4ded-9364-1acd27821b9b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:37.722495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a062-fd59-4ded-9364-1acd27821b9b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:37.723923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 649/1293] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:37.729273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691f1f9-7f51-4233-b82e-35ebdb933ef0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:37.729682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691f1f9-7f51-4233-b82e-35ebdb933ef0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:37.729837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691f1f9-7f51-4233-b82e-35ebdb933ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:37.746402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f691f1f9-7f51-4233-b82e-35ebdb933ef0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:37.754148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f691f1f9-7f51-4233-b82e-35ebdb933ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:37.777453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:37.813560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 645/1294] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:37.837239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a062-fd59-4ded-9364-1acd27821b9b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:37.867742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:37.936311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a062-fd59-4ded-9364-1acd27821b9b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:37.970123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b102181-ae6c-4340-9a7b-2c55081e1b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:37.970784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102181-ae6c-4340-9a7b-2c55081e1b57 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:37.971072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102181-ae6c-4340-9a7b-2c55081e1b57 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:37.971384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102181-ae6c-4340-9a7b-2c55081e1b57 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:37.972147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b102181-ae6c-4340-9a7b-2c55081e1b57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:37.973462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b102181-ae6c-4340-9a7b-2c55081e1b57 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:38.085263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 650/1295] 10.222.0.98 () {60 vars in 1141 bytes} [Fri Jul 15 15:36:37 2022] POST /identity/v3/services => generated 246 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:36:40.133605 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a634fd2-184a-4950-b312-a3e89a90b41b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:40.134190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a634fd2-184a-4950-b312-a3e89a90b41b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:40.134304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a634fd2-184a-4950-b312-a3e89a90b41b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:40.135850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 646/1296] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:40.141097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41285732-fe62-4f42-b018-167414a33e07 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:40.141491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41285732-fe62-4f42-b018-167414a33e07 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:40.141663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41285732-fe62-4f42-b018-167414a33e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:40.159196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-41285732-fe62-4f42-b018-167414a33e07 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:40.160565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-41285732-fe62-4f42-b018-167414a33e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:40.231069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 651/1297] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:40.254554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a634fd2-184a-4950-b312-a3e89a90b41b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:40.354315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a634fd2-184a-4950-b312-a3e89a90b41b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:40.389035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:40.389762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:40.390051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:40.390354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:40.391003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:40.392423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:40.404007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:40.405134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:40.406027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 647/1298] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:36:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:40.411328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41285732-fe62-4f42-b018-167414a33e07 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:40.429897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:40.507713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41285732-fe62-4f42-b018-167414a33e07 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:40.542765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:40.543938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:40.544310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:40.544599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:40.549407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:40.550229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:40.551694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:40.558359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 652/1299] 10.222.0.98 () {56 vars in 1116 bytes} [Fri Jul 15 15:36:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:40.564852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:40.597840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:40.669566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da9bc6-66ee-4a5d-8d1a-225928cb6974 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:40.719486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbddb11-cb9d-4870-9f02-5817427d75db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:40.720632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbddb11-cb9d-4870-9f02-5817427d75db None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:40.721112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbddb11-cb9d-4870-9f02-5817427d75db None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:40.721616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbddb11-cb9d-4870-9f02-5817427d75db None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:40.723615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbddb11-cb9d-4870-9f02-5817427d75db None admin] RBAC: Adding query filter params (interface=public, service_id=fb14ffbd512c49ebb8cb17eba13d2388, region_id=RegionOne) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:40.724142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbddb11-cb9d-4870-9f02-5817427d75db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:40.726619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbddb11-cb9d-4870-9f02-5817427d75db None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:40.734923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 648/1300] 10.222.0.98 () {56 vars in 1251 bytes} [Fri Jul 15 15:36:40 2022] GET /identity/v3/endpoints?service_id=fb14ffbd512c49ebb8cb17eba13d2388&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:40.743040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:40.828497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:40.846216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0b4ae-95c8-4c01-80e7-630a05c1f5f5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:40.880545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398499ec-f23f-4568-9e8f-85517a4e16b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:40.881327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398499ec-f23f-4568-9e8f-85517a4e16b4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:40.881611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398499ec-f23f-4568-9e8f-85517a4e16b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:40.881884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398499ec-f23f-4568-9e8f-85517a4e16b4 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:40.882547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398499ec-f23f-4568-9e8f-85517a4e16b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:40.883956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398499ec-f23f-4568-9e8f-85517a4e16b4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:40.890984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 653/1301] 10.222.0.98 () {56 vars in 1087 bytes} [Fri Jul 15 15:36:40 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:36:42.918895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db082cd9-816d-4cd3-8c26-ca4285817f4e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:42.919442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db082cd9-816d-4cd3-8c26-ca4285817f4e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:42.919566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db082cd9-816d-4cd3-8c26-ca4285817f4e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:42.920881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 649/1302] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:36:42.926051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912209e8-4d78-4f4a-a0f4-a05e05e5ca00 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:42.926437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912209e8-4d78-4f4a-a0f4-a05e05e5ca00 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:42.926635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912209e8-4d78-4f4a-a0f4-a05e05e5ca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:42.943469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-912209e8-4d78-4f4a-a0f4-a05e05e5ca00 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:42.946258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-912209e8-4d78-4f4a-a0f4-a05e05e5ca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:43.009708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 654/1303] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:36:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:36:43.032713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db082cd9-816d-4cd3-8c26-ca4285817f4e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:43.126817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:43.134127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db082cd9-816d-4cd3-8c26-ca4285817f4e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:43.167119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:43.167936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:43.168223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:43.168502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:43.169190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:43.170509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:36:43.182025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:36:43.189209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jul 15 15:36:43.190054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 650/1304] 10.222.0.98 () {56 vars in 1117 bytes} [Fri Jul 15 15:36:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:36:43.190843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912209e8-4d78-4f4a-a0f4-a05e05e5ca00 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:43.264578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:43.287024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912209e8-4d78-4f4a-a0f4-a05e05e5ca00 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:43.320678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7f700-7d04-4771-bb1a-2a0cdd74e179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:43.321334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f700-7d04-4771-bb1a-2a0cdd74e179 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:43.321639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f700-7d04-4771-bb1a-2a0cdd74e179 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:43.321918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7f700-7d04-4771-bb1a-2a0cdd74e179 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:43.322606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7f700-7d04-4771-bb1a-2a0cdd74e179 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:43.322913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7f700-7d04-4771-bb1a-2a0cdd74e179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:43.324335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7f700-7d04-4771-bb1a-2a0cdd74e179 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:43.329446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 655/1305] 10.222.0.98 () {56 vars in 1116 bytes} [Fri Jul 15 15:36:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:43.335474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:43.471064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d1e6c-3117-4781-8576-0d954f1a76d3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:43.504930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3f02d-1ea3-40ba-8ee8-1d4238e51a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:43.505587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3f02d-1ea3-40ba-8ee8-1d4238e51a05 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:43.505876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3f02d-1ea3-40ba-8ee8-1d4238e51a05 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:43.506156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3f02d-1ea3-40ba-8ee8-1d4238e51a05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:43.506888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3f02d-1ea3-40ba-8ee8-1d4238e51a05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:43.508300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3f02d-1ea3-40ba-8ee8-1d4238e51a05 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:43.586114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 651/1306] 10.222.0.98 () {60 vars in 1144 bytes} [Fri Jul 15 15:36:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:36:53.334547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=30 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:53.581615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:32.753785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58f02-214c-4e0c-8863-35479767682f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:32.754973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58f02-214c-4e0c-8863-35479767682f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:32.755508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58f02-214c-4e0c-8863-35479767682f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:32.762623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c34a1-2609-4a49-b6be-85f3a39f40f9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:32.764280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c34a1-2609-4a49-b6be-85f3a39f40f9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:32.764280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c34a1-2609-4a49-b6be-85f3a39f40f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:32.766447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 656/1307] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:32.841588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2a0c34a1-2609-4a49-b6be-85f3a39f40f9 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:32.843167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2a0c34a1-2609-4a49-b6be-85f3a39f40f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:32.995887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 652/1308] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:44:33.040746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87455a-9358-422a-80eb-cd6c08b3f3c4 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:33.041442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87455a-9358-422a-80eb-cd6c08b3f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:33.042257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87455a-9358-422a-80eb-cd6c08b3f3c4 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:33.045913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 657/1309] 10.222.0.98 () {54 vars in 964 bytes} [Fri Jul 15 15:44:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:33.052383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b0e1-5d29-41a1-bd1f-23bc529bfc81 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:33.052642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b0e1-5d29-41a1-bd1f-23bc529bfc81 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:33.052932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94b0e1-5d29-41a1-bd1f-23bc529bfc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:33.161950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dd94b0e1-5d29-41a1-bd1f-23bc529bfc81 None None] MFA Rules not processed for user `9d20de9c921942d1afeb63e87272f782`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:33.163773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dd94b0e1-5d29-41a1-bd1f-23bc529bfc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:33.191384 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:33.274680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 653/1310] 10.222.0.98 () {58 vars in 1061 bytes} [Fri Jul 15 15:44:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:33.284147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87455a-9358-422a-80eb-cd6c08b3f3c4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:33.286084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4c87455a-9358-422a-80eb-cd6c08b3f3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:33.426922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87455a-9358-422a-80eb-cd6c08b3f3c4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:33.504451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f28cae-16a4-4a7d-ab29-92f9e1a61907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:33.505110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28cae-16a4-4a7d-ab29-92f9e1a61907 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:33.505395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28cae-16a4-4a7d-ab29-92f9e1a61907 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:33.505670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28cae-16a4-4a7d-ab29-92f9e1a61907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:33.507315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d1f28cae-16a4-4a7d-ab29-92f9e1a61907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:33.567270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:33.625515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f28cae-16a4-4a7d-ab29-92f9e1a61907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:33.627166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f28cae-16a4-4a7d-ab29-92f9e1a61907 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:33.650047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 658/1311] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:44:35.965319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d020dec-b3b9-4e10-9ad9-0e95964571d6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:35.965895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d020dec-b3b9-4e10-9ad9-0e95964571d6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:35.965895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d020dec-b3b9-4e10-9ad9-0e95964571d6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:35.967598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 654/1312] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:35.973225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8da758-9da0-468f-9f2b-676feec8ee9e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:35.973737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8da758-9da0-468f-9f2b-676feec8ee9e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:35.973737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8da758-9da0-468f-9f2b-676feec8ee9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:36.065698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7f8da758-9da0-468f-9f2b-676feec8ee9e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:36.066959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7f8da758-9da0-468f-9f2b-676feec8ee9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:36.168934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 659/1313] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:36.207143 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d020dec-b3b9-4e10-9ad9-0e95964571d6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:36.208718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8d020dec-b3b9-4e10-9ad9-0e95964571d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:36.318116 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d020dec-b3b9-4e10-9ad9-0e95964571d6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:36.392615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:36.395435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4082e2-a1cc-455a-9a46-fefae3a8ef1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:36.396182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4082e2-a1cc-455a-9a46-fefae3a8ef1f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:36.396556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4082e2-a1cc-455a-9a46-fefae3a8ef1f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:36.396961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4082e2-a1cc-455a-9a46-fefae3a8ef1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:36.398815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9b4082e2-a1cc-455a-9a46-fefae3a8ef1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:36.516330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4082e2-a1cc-455a-9a46-fefae3a8ef1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:36.517790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4082e2-a1cc-455a-9a46-fefae3a8ef1f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:36.535714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 655/1314] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:39.271814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac967-6711-4dc0-a026-2ab38b88178f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:39.272403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac967-6711-4dc0-a026-2ab38b88178f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:39.272403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac967-6711-4dc0-a026-2ab38b88178f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:39.273745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 660/1315] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:39.279429 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bc1cc-2a6f-4983-9084-260087209504 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:39.279838 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bc1cc-2a6f-4983-9084-260087209504 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:39.279982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73bc1cc-2a6f-4983-9084-260087209504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:39.343109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f73bc1cc-2a6f-4983-9084-260087209504 None None] MFA Rules not processed for user `c956159832a6436bbf4c4a3517bd6170`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:39.344252 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f73bc1cc-2a6f-4983-9084-260087209504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:39.396687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:39.440624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 656/1316] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:39 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:39.480467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ac967-6711-4dc0-a026-2ab38b88178f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:39.482043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e47ac967-6711-4dc0-a026-2ab38b88178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:39.491907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:39.588558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ac967-6711-4dc0-a026-2ab38b88178f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:39.669126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b99c69-98e3-4e58-b95b-6ecaa6fa6f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:39.670492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b99c69-98e3-4e58-b95b-6ecaa6fa6f87 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:39.671004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b99c69-98e3-4e58-b95b-6ecaa6fa6f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:39.671557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b99c69-98e3-4e58-b95b-6ecaa6fa6f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:39.674484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c9b99c69-98e3-4e58-b95b-6ecaa6fa6f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:39.739893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:39.788870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b99c69-98e3-4e58-b95b-6ecaa6fa6f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:39.790340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b99c69-98e3-4e58-b95b-6ecaa6fa6f87 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:39.810596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 661/1317] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:42.643129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6bdd6-2f8c-4b23-ad88-7dfadf58c160 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:42.643730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6bdd6-2f8c-4b23-ad88-7dfadf58c160 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:42.643850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6bdd6-2f8c-4b23-ad88-7dfadf58c160 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:42.645649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 657/1318] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:42.651943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf89f-427b-4951-9cf3-265cd2bc2e9c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:42.652383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf89f-427b-4951-9cf3-265cd2bc2e9c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:42.652478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbf89f-427b-4951-9cf3-265cd2bc2e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:42.743723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bbbbf89f-427b-4951-9cf3-265cd2bc2e9c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:42.744969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bbbbf89f-427b-4951-9cf3-265cd2bc2e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:42.844800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 662/1319] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:42.883054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6bdd6-2f8c-4b23-ad88-7dfadf58c160 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:42.884633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f8b6bdd6-2f8c-4b23-ad88-7dfadf58c160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:42.987239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6bdd6-2f8c-4b23-ad88-7dfadf58c160 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:43.061757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2589-0a51-458e-ab64-9309eeaa84ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:43.062408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2589-0a51-458e-ab64-9309eeaa84ab service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:43.062698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2589-0a51-458e-ab64-9309eeaa84ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:43.062977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2589-0a51-458e-ab64-9309eeaa84ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:43.064648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-59bb2589-0a51-458e-ab64-9309eeaa84ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:43.072051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:43.172357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2589-0a51-458e-ab64-9309eeaa84ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:43.173823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2589-0a51-458e-ab64-9309eeaa84ab service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:43.192411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 658/1320] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:46.243832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67947c16-90e1-4561-9591-6aaa7b6b73a8 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:46.244368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67947c16-90e1-4561-9591-6aaa7b6b73a8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:46.244482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67947c16-90e1-4561-9591-6aaa7b6b73a8 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:46.245787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 663/1321] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:46.250987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cfe01-c04e-4bfc-8a19-51f3ee18f0e6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:46.251417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cfe01-c04e-4bfc-8a19-51f3ee18f0e6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:46.251612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cfe01-c04e-4bfc-8a19-51f3ee18f0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:46.316789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7e3cfe01-c04e-4bfc-8a19-51f3ee18f0e6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:46.318006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7e3cfe01-c04e-4bfc-8a19-51f3ee18f0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:46.375799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:46.419537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 659/1322] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:46.460600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67947c16-90e1-4561-9591-6aaa7b6b73a8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:46.462165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-67947c16-90e1-4561-9591-6aaa7b6b73a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:46.483853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:46.580704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67947c16-90e1-4561-9591-6aaa7b6b73a8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:46.660470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3aed9-54df-453a-b89b-4de7c91ef132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:46.661129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3aed9-54df-453a-b89b-4de7c91ef132 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:46.661423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3aed9-54df-453a-b89b-4de7c91ef132 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:46.661705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3aed9-54df-453a-b89b-4de7c91ef132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:46.663388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f4f3aed9-54df-453a-b89b-4de7c91ef132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:46.735215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:46.772143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3aed9-54df-453a-b89b-4de7c91ef132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:46.773604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3aed9-54df-453a-b89b-4de7c91ef132 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:46.792950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 664/1323] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:49.001497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c44fa-78cb-4b7a-8399-7ca06320ca70 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:49.002002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c44fa-78cb-4b7a-8399-7ca06320ca70 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:49.002002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c44fa-78cb-4b7a-8399-7ca06320ca70 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:49.003582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 660/1324] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:49.008630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7203d8-baa4-4d56-9157-7a99c3721a60 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:49.009011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7203d8-baa4-4d56-9157-7a99c3721a60 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:49.009190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7203d8-baa4-4d56-9157-7a99c3721a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:49.078519 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ba7203d8-baa4-4d56-9157-7a99c3721a60 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:49.079759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ba7203d8-baa4-4d56-9157-7a99c3721a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:49.176870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 665/1325] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:49.213316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c44fa-78cb-4b7a-8399-7ca06320ca70 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:49.214872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-576c44fa-78cb-4b7a-8399-7ca06320ca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:49.327194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c44fa-78cb-4b7a-8399-7ca06320ca70 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:49.406544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8846137-8aee-4b94-a216-4cad0ec699a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:49.407212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846137-8aee-4b94-a216-4cad0ec699a0 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:49.407540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846137-8aee-4b94-a216-4cad0ec699a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:49.407817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846137-8aee-4b94-a216-4cad0ec699a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:49.409510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c8846137-8aee-4b94-a216-4cad0ec699a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:49.427666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:49.519985 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8846137-8aee-4b94-a216-4cad0ec699a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:49.521439 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8846137-8aee-4b94-a216-4cad0ec699a0 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:49.539880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 661/1326] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:51.643853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becda639-21d7-48cb-b4da-6b687e4acbbd None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:51.644411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becda639-21d7-48cb-b4da-6b687e4acbbd None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:51.644411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becda639-21d7-48cb-b4da-6b687e4acbbd None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:51.646112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 666/1327] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:51.651615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f49fd5-3f38-415a-9158-ebbee47914de None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:51.652035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f49fd5-3f38-415a-9158-ebbee47914de None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:51.652333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f49fd5-3f38-415a-9158-ebbee47914de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:51.719828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-22f49fd5-3f38-415a-9158-ebbee47914de None None] MFA Rules not processed for user `c956159832a6436bbf4c4a3517bd6170`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:51.721064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-22f49fd5-3f38-415a-9158-ebbee47914de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:51.788120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:51.823577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 662/1328] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:51 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:51.861792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becda639-21d7-48cb-b4da-6b687e4acbbd None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:51.863423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-becda639-21d7-48cb-b4da-6b687e4acbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:51.894428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:51.967093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becda639-21d7-48cb-b4da-6b687e4acbbd None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:52.040680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f1e64-b9d9-4980-8854-5a8cbd9f2dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:52.041347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f1e64-b9d9-4980-8854-5a8cbd9f2dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:52.041623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f1e64-b9d9-4980-8854-5a8cbd9f2dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:52.041904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f1e64-b9d9-4980-8854-5a8cbd9f2dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:52.043585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e53f1e64-b9d9-4980-8854-5a8cbd9f2dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:52.139543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:52.171056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f1e64-b9d9-4980-8854-5a8cbd9f2dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:52.173069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f1e64-b9d9-4980-8854-5a8cbd9f2dc9 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:52.193351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 667/1329] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:51 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:54.741433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827bf8-8486-4e40-bba5-4cf27d0aeb7b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:54.741991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827bf8-8486-4e40-bba5-4cf27d0aeb7b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:54.741991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827bf8-8486-4e40-bba5-4cf27d0aeb7b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:54.743470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 663/1330] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:54.749457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc37e5-7053-4315-8f41-1babef98e778 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:54.749862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc37e5-7053-4315-8f41-1babef98e778 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:54.750022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc37e5-7053-4315-8f41-1babef98e778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:54.814601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-29dc37e5-7053-4315-8f41-1babef98e778 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:54.815845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-29dc37e5-7053-4315-8f41-1babef98e778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:54.911249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 668/1331] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:54.947511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86827bf8-8486-4e40-bba5-4cf27d0aeb7b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:54.949041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-86827bf8-8486-4e40-bba5-4cf27d0aeb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:55.052621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86827bf8-8486-4e40-bba5-4cf27d0aeb7b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:55.126598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e38e63-e754-499d-9d5f-5f09a6b314ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:55.127244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38e63-e754-499d-9d5f-5f09a6b314ba service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:55.127557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38e63-e754-499d-9d5f-5f09a6b314ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:55.127834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38e63-e754-499d-9d5f-5f09a6b314ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:55.129489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-81e38e63-e754-499d-9d5f-5f09a6b314ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:55.156101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:55.236736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e38e63-e754-499d-9d5f-5f09a6b314ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:55.238188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e38e63-e754-499d-9d5f-5f09a6b314ba service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:55.255684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 664/1332] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:57.304640 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06594-6a03-447d-b955-f5d8f095f197 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:57.305427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06594-6a03-447d-b955-f5d8f095f197 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:57.305427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06594-6a03-447d-b955-f5d8f095f197 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:57.306808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 669/1333] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:44:57.311948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989403b4-3ecd-4616-8b6e-f7aafebf6b05 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:57.312344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989403b4-3ecd-4616-8b6e-f7aafebf6b05 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:57.312517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989403b4-3ecd-4616-8b6e-f7aafebf6b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:57.375975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-989403b4-3ecd-4616-8b6e-f7aafebf6b05 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:57.377177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-989403b4-3ecd-4616-8b6e-f7aafebf6b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:57.452427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:57.475108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 665/1334] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:57.516768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd06594-6a03-447d-b955-f5d8f095f197 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:57.518316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5fd06594-6a03-447d-b955-f5d8f095f197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:57.588626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:57.650859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd06594-6a03-447d-b955-f5d8f095f197 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:57.725769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a99f25-ed16-4a55-be11-f620d327f575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:57.726415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99f25-ed16-4a55-be11-f620d327f575 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:57.726701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99f25-ed16-4a55-be11-f620d327f575 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:57.726975 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99f25-ed16-4a55-be11-f620d327f575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:57.728630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-43a99f25-ed16-4a55-be11-f620d327f575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:57.809467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:57.846947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a99f25-ed16-4a55-be11-f620d327f575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:57.848351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a99f25-ed16-4a55-be11-f620d327f575 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:57.866889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 670/1335] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:00.579578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1437f-8572-42ba-8383-19cb758b463c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:00.580099 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1437f-8572-42ba-8383-19cb758b463c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:00.580225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1437f-8572-42ba-8383-19cb758b463c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:00.581543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 666/1336] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:00.586943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d909e3-388c-4ccd-bf34-377f6a4edc2a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:00.587383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d909e3-388c-4ccd-bf34-377f6a4edc2a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:00.587542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d909e3-388c-4ccd-bf34-377f6a4edc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:00.658660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-68d909e3-388c-4ccd-bf34-377f6a4edc2a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:00.659833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-68d909e3-388c-4ccd-bf34-377f6a4edc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:00.759038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 671/1337] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:00.795412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1437f-8572-42ba-8383-19cb758b463c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:00.796934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-09e1437f-8572-42ba-8383-19cb758b463c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:00.899210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1437f-8572-42ba-8383-19cb758b463c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:00.975654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468a034-c4d1-4d88-b7fe-d5bdee968956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:00.976306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468a034-c4d1-4d88-b7fe-d5bdee968956 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:00.976590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468a034-c4d1-4d88-b7fe-d5bdee968956 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:00.976865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468a034-c4d1-4d88-b7fe-d5bdee968956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:00.978510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c468a034-c4d1-4d88-b7fe-d5bdee968956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:01.014736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:01.087551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468a034-c4d1-4d88-b7fe-d5bdee968956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:01.089010 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468a034-c4d1-4d88-b7fe-d5bdee968956 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:01.106534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 667/1338] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:05.280492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b64c-04c7-4cb4-9b46-246a9041ddb4 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:05.281025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b64c-04c7-4cb4-9b46-246a9041ddb4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:05.281025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3b64c-04c7-4cb4-9b46-246a9041ddb4 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:05.282499 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 672/1339] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:05.287681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02017bc5-ff60-4db0-ab71-46fa8fa3f42e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:05.288191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02017bc5-ff60-4db0-ab71-46fa8fa3f42e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:05.288284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02017bc5-ff60-4db0-ab71-46fa8fa3f42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:05.354379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-02017bc5-ff60-4db0-ab71-46fa8fa3f42e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:05.355672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-02017bc5-ff60-4db0-ab71-46fa8fa3f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:05.450843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:05.463343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 668/1340] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:05.515028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aad7e-c857-46c7-9c9a-137f8bec1070 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:05.515354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aad7e-c857-46c7-9c9a-137f8bec1070 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:05.515655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aad7e-c857-46c7-9c9a-137f8bec1070 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:05.517066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 673/1341] 10.222.0.98 () {54 vars in 964 bytes} [Fri Jul 15 15:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:45:05.523021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414e30-894d-4e71-997f-85d021280c32 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:05.523337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414e30-894d-4e71-997f-85d021280c32 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:05.523624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38414e30-894d-4e71-997f-85d021280c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:05.599419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-38414e30-894d-4e71-997f-85d021280c32 None None] MFA Rules not processed for user `9d20de9c921942d1afeb63e87272f782`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:05.600556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-38414e30-894d-4e71-997f-85d021280c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:05.678510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 669/1342] 10.222.0.98 () {58 vars in 1061 bytes} [Fri Jul 15 15:45:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:05.688184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81aad7e-c857-46c7-9c9a-137f8bec1070 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:05.689725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d81aad7e-c857-46c7-9c9a-137f8bec1070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:05.737732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:05.793820 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81aad7e-c857-46c7-9c9a-137f8bec1070 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:05.869966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8279ed-d04d-4888-b67a-b3219d004996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:05.870567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8279ed-d04d-4888-b67a-b3219d004996 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:05.870820 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8279ed-d04d-4888-b67a-b3219d004996 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:05.871097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8279ed-d04d-4888-b67a-b3219d004996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:05.873363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9d8279ed-d04d-4888-b67a-b3219d004996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:05.981283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:05.997553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8279ed-d04d-4888-b67a-b3219d004996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:05.999007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8279ed-d04d-4888-b67a-b3219d004996 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:06.018076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 674/1343] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:09.649496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abb344-db1e-4536-a343-ed87b7966090 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:09.650079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abb344-db1e-4536-a343-ed87b7966090 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:09.650186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abb344-db1e-4536-a343-ed87b7966090 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:09.651566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 670/1344] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:09.657729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166121a-b1b9-4be7-aa9d-2a55b0bbe979 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:09.658270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166121a-b1b9-4be7-aa9d-2a55b0bbe979 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:09.658411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166121a-b1b9-4be7-aa9d-2a55b0bbe979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:09.725710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f166121a-b1b9-4be7-aa9d-2a55b0bbe979 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:09.726925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f166121a-b1b9-4be7-aa9d-2a55b0bbe979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:09.838885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 675/1345] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:09.877564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abb344-db1e-4536-a343-ed87b7966090 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:09.881507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-16abb344-db1e-4536-a343-ed87b7966090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:09.969371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:09.987659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abb344-db1e-4536-a343-ed87b7966090 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:10.065424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cff45-9029-4d5b-92cf-024a1e320eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:10.066081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cff45-9029-4d5b-92cf-024a1e320eba service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:10.066372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cff45-9029-4d5b-92cf-024a1e320eba service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:10.066655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cff45-9029-4d5b-92cf-024a1e320eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:10.068402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-029cff45-9029-4d5b-92cf-024a1e320eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:10.220719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cff45-9029-4d5b-92cf-024a1e320eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:10.222383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cff45-9029-4d5b-92cf-024a1e320eba service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:10.243750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 671/1346] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:45:14.982975 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61875c2-8b12-41a2-b8bc-230a31b8ddd2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:14.983548 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61875c2-8b12-41a2-b8bc-230a31b8ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:14.983653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61875c2-8b12-41a2-b8bc-230a31b8ddd2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:14.984970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 676/1347] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:14.990305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0e0eb-fd45-484b-b90d-4dee0b86a155 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:14.991122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0e0eb-fd45-484b-b90d-4dee0b86a155 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:14.991441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0e0eb-fd45-484b-b90d-4dee0b86a155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:15.006072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:15.061319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-10c0e0eb-fd45-484b-b90d-4dee0b86a155 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:15.062565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-10c0e0eb-fd45-484b-b90d-4dee0b86a155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:15.164541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 672/1348] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:15.202929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61875c2-8b12-41a2-b8bc-230a31b8ddd2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:15.204487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c61875c2-8b12-41a2-b8bc-230a31b8ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:15.262739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:15.312453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61875c2-8b12-41a2-b8bc-230a31b8ddd2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:15.388510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e59fb-b81d-48b0-ba11-8943c462a406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:15.389728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e59fb-b81d-48b0-ba11-8943c462a406 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:15.390014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e59fb-b81d-48b0-ba11-8943c462a406 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:15.390305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e59fb-b81d-48b0-ba11-8943c462a406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:15.391973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1c6e59fb-b81d-48b0-ba11-8943c462a406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:15.499489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:15.502964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e59fb-b81d-48b0-ba11-8943c462a406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:15.504442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e59fb-b81d-48b0-ba11-8943c462a406 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:15.522441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 677/1349] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:20.410872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e32f-0904-45f4-a19b-13c3ecc8f4de None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:20.411393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e32f-0904-45f4-a19b-13c3ecc8f4de None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:20.411494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e32f-0904-45f4-a19b-13c3ecc8f4de None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:20.413040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 673/1350] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:20.418513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a402d99-acae-40ac-aa3a-700244dddb8c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:20.418938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a402d99-acae-40ac-aa3a-700244dddb8c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:20.419067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a402d99-acae-40ac-aa3a-700244dddb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:20.544728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9a402d99-acae-40ac-aa3a-700244dddb8c None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:20.546268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9a402d99-acae-40ac-aa3a-700244dddb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:20.668868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 678/1351] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:45:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:24.837274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6092-3b55-4a1b-b5a2-06022a0f012a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:24.837858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6092-3b55-4a1b-b5a2-06022a0f012a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:24.837858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6092-3b55-4a1b-b5a2-06022a0f012a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:24.839419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 674/1352] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:24.844540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc8176-3fa5-40e0-9bb3-801dfcce22ad None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:24.844973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc8176-3fa5-40e0-9bb3-801dfcce22ad None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:24.845150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc8176-3fa5-40e0-9bb3-801dfcce22ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:24.902031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:24.925813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-51bc8176-3fa5-40e0-9bb3-801dfcce22ad None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:24.927043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-51bc8176-3fa5-40e0-9bb3-801dfcce22ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:25.046464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 679/1353] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:45:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:25.162673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=59 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:26.352010 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03a27d-d689-42b2-9df7-dff37e007302 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:26.352627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03a27d-d689-42b2-9df7-dff37e007302 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:26.352627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03a27d-d689-42b2-9df7-dff37e007302 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:26.353889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 675/1354] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:26.359671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e08675-4fcc-4216-9de5-6f612b40bbfa None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:26.360087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e08675-4fcc-4216-9de5-6f612b40bbfa None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:26.360265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e08675-4fcc-4216-9de5-6f612b40bbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:26.443820 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-20e08675-4fcc-4216-9de5-6f612b40bbfa None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:26.445046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-20e08675-4fcc-4216-9de5-6f612b40bbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:26.489426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:26.555235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 680/1355] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:45:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:29.981051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f742-5f58-4d99-ad74-ba05cec8fa3e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:29.981985 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f742-5f58-4d99-ad74-ba05cec8fa3e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:29.982423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f742-5f58-4d99-ad74-ba05cec8fa3e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:29.986056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 676/1356] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:29.990728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479e8a0-4368-47fc-beb0-f74ac7ca4c12 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:29.992425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479e8a0-4368-47fc-beb0-f74ac7ca4c12 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:29.992893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479e8a0-4368-47fc-beb0-f74ac7ca4c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:30.071497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2479e8a0-4368-47fc-beb0-f74ac7ca4c12 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:30.072711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2479e8a0-4368-47fc-beb0-f74ac7ca4c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:30.177049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 681/1357] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:30.232729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729bfe5-ea31-4a3f-8dba-0b737d4f1749 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:30.233310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729bfe5-ea31-4a3f-8dba-0b737d4f1749 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:30.234831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729bfe5-ea31-4a3f-8dba-0b737d4f1749 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:30.236955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 677/1358] 10.222.0.98 () {54 vars in 961 bytes} [Fri Jul 15 15:45:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:30.245830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b74b-662b-4431-9dcf-980df4aeced3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:30.246617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b74b-662b-4431-9dcf-980df4aeced3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:30.250919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b74b-662b-4431-9dcf-980df4aeced3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:30.263366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:30.330153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a619b74b-662b-4431-9dcf-980df4aeced3 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:30.331540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a619b74b-662b-4431-9dcf-980df4aeced3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:30.462991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 682/1359] 10.222.0.98 () {58 vars in 1058 bytes} [Fri Jul 15 15:45:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:45:30.471087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729bfe5-ea31-4a3f-8dba-0b737d4f1749 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:30.472743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c729bfe5-ea31-4a3f-8dba-0b737d4f1749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:30.625188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729bfe5-ea31-4a3f-8dba-0b737d4f1749 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:30.702840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:30.748236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ada808-50af-4774-8d40-1c77fe69ac41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:30.748953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada808-50af-4774-8d40-1c77fe69ac41 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:30.749305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada808-50af-4774-8d40-1c77fe69ac41 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:30.749651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada808-50af-4774-8d40-1c77fe69ac41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:30.751399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d5ada808-50af-4774-8d40-1c77fe69ac41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:30.845279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ada808-50af-4774-8d40-1c77fe69ac41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:30.846997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ada808-50af-4774-8d40-1c77fe69ac41 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:30.867205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 678/1360] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:33.113177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188043-1d33-47d7-bd22-d3a5a5a4d90e None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:33.113708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188043-1d33-47d7-bd22-d3a5a5a4d90e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:33.113708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188043-1d33-47d7-bd22-d3a5a5a4d90e None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:33.115202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 683/1361] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:33.120354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532c21c-07ea-484e-b5af-1a04ded029d1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:33.120758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532c21c-07ea-484e-b5af-1a04ded029d1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:33.120924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532c21c-07ea-484e-b5af-1a04ded029d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:33.193159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e532c21c-07ea-484e-b5af-1a04ded029d1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:33.194556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e532c21c-07ea-484e-b5af-1a04ded029d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:33.235665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:33.302196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 679/1362] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:33.348042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba188043-1d33-47d7-bd22-d3a5a5a4d90e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:33.350293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ba188043-1d33-47d7-bd22-d3a5a5a4d90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:33.435662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:33.528077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba188043-1d33-47d7-bd22-d3a5a5a4d90e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:33.655097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade18110-f2ef-4fff-8417-ab2b0ea6e863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:33.656137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade18110-f2ef-4fff-8417-ab2b0ea6e863 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:33.656609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade18110-f2ef-4fff-8417-ab2b0ea6e863 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:33.656939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade18110-f2ef-4fff-8417-ab2b0ea6e863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:33.659578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ade18110-f2ef-4fff-8417-ab2b0ea6e863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:33.710995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:33.787093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade18110-f2ef-4fff-8417-ab2b0ea6e863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:33.788754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade18110-f2ef-4fff-8417-ab2b0ea6e863 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:33.808526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 684/1363] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:36.299831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18713a4f-5835-4a47-9fb7-0e3b5e04737e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:36.300394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18713a4f-5835-4a47-9fb7-0e3b5e04737e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:36.300394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18713a4f-5835-4a47-9fb7-0e3b5e04737e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:36.301794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 680/1364] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:36.308014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703f2a4-7b15-4dfc-8454-2d405e50c82f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:36.308509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703f2a4-7b15-4dfc-8454-2d405e50c82f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:36.308601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703f2a4-7b15-4dfc-8454-2d405e50c82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:36.399224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f703f2a4-7b15-4dfc-8454-2d405e50c82f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:36.400511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f703f2a4-7b15-4dfc-8454-2d405e50c82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:36.518216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:36.525928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 685/1365] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:36.570380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff44d17-0096-4071-985e-a441cb6e51b0 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:36.570673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff44d17-0096-4071-985e-a441cb6e51b0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:36.570982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff44d17-0096-4071-985e-a441cb6e51b0 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:36.572522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 681/1366] 10.222.0.98 () {54 vars in 961 bytes} [Fri Jul 15 15:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:36.578555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e134d87-8ad3-4f1c-b281-4a448d7bc4d3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:36.578858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e134d87-8ad3-4f1c-b281-4a448d7bc4d3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:36.579144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e134d87-8ad3-4f1c-b281-4a448d7bc4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:36.660387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1e134d87-8ad3-4f1c-b281-4a448d7bc4d3 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:36.661625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1e134d87-8ad3-4f1c-b281-4a448d7bc4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:36.785883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 686/1367] 10.222.0.98 () {58 vars in 1058 bytes} [Fri Jul 15 15:45:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:36.793846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff44d17-0096-4071-985e-a441cb6e51b0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:36.795438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9ff44d17-0096-4071-985e-a441cb6e51b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:36.958952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff44d17-0096-4071-985e-a441cb6e51b0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:36.980249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:37.082285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece8d33-7896-4fa6-9eb2-64769991da29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:37.082932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece8d33-7896-4fa6-9eb2-64769991da29 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:37.083221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece8d33-7896-4fa6-9eb2-64769991da29 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:37.083535 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece8d33-7896-4fa6-9eb2-64769991da29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:37.085197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8ece8d33-7896-4fa6-9eb2-64769991da29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:37.183817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece8d33-7896-4fa6-9eb2-64769991da29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:37.185551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece8d33-7896-4fa6-9eb2-64769991da29 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:37.204269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 682/1368] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:38.217964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab947c75-ad9f-4cd1-ae3f-99c2e1b3a807 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:38.218517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab947c75-ad9f-4cd1-ae3f-99c2e1b3a807 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:38.218517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab947c75-ad9f-4cd1-ae3f-99c2e1b3a807 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:38.219978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 687/1369] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:38.225287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f122-ac81-477d-b298-3a2374330a3b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:38.225700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f122-ac81-477d-b298-3a2374330a3b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:38.225883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f122-ac81-477d-b298-3a2374330a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:38.258295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:38.294975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5771f122-ac81-477d-b298-3a2374330a3b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:38.296272 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5771f122-ac81-477d-b298-3a2374330a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:38.401270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 683/1370] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:38.856299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce8f9-f4d2-4d5d-b260-477323f933ea None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:38.856299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce8f9-f4d2-4d5d-b260-477323f933ea None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:38.856299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ce8f9-f4d2-4d5d-b260-477323f933ea None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:38.856299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 688/1371] 10.222.0.98 () {54 vars in 963 bytes} [Fri Jul 15 15:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:38.857042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff8bad-a41a-4edd-899c-8832724a1ac6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:38.857042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff8bad-a41a-4edd-899c-8832724a1ac6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:38.857042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff8bad-a41a-4edd-899c-8832724a1ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:38.857042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-97ff8bad-a41a-4edd-899c-8832724a1ac6 None None] MFA Rules not processed for user `68015ac9febe42ef917ca58fae7b1c26`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:38.857042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-97ff8bad-a41a-4edd-899c-8832724a1ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:38.910582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:38.941848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 684/1372] 10.222.0.98 () {58 vars in 1060 bytes} [Fri Jul 15 15:45:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:38.949244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ce8f9-f4d2-4d5d-b260-477323f933ea None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:38.950741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9b6ce8f9-f4d2-4d5d-b260-477323f933ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:39.005880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:39.062151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ce8f9-f4d2-4d5d-b260-477323f933ea None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:39.145126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b334a5-74f8-4445-8f94-6623e4199c6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:39.145697 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b334a5-74f8-4445-8f94-6623e4199c6d service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:39.145992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b334a5-74f8-4445-8f94-6623e4199c6d service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:39.146266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b334a5-74f8-4445-8f94-6623e4199c6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:39.147938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-96b334a5-74f8-4445-8f94-6623e4199c6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:39.256133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:39.269755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b334a5-74f8-4445-8f94-6623e4199c6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:39.271249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b334a5-74f8-4445-8f94-6623e4199c6d service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:39.291710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 689/1373] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:39.534481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c4b0b-ad4a-4e5f-aaab-0b740a174128 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:39.534481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c4b0b-ad4a-4e5f-aaab-0b740a174128 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:39.534481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c4b0b-ad4a-4e5f-aaab-0b740a174128 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:39.534481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 685/1374] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:39.539640 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3441f80-a492-4be3-9789-fd72d9df0fb1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:39.540260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3441f80-a492-4be3-9789-fd72d9df0fb1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:39.540729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3441f80-a492-4be3-9789-fd72d9df0fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:39.618896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a3441f80-a492-4be3-9789-fd72d9df0fb1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:39.620196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a3441f80-a492-4be3-9789-fd72d9df0fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:39.724571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 690/1375] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:39.769471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c4b0b-ad4a-4e5f-aaab-0b740a174128 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:39.771081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-538c4b0b-ad4a-4e5f-aaab-0b740a174128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:39.921391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c4b0b-ad4a-4e5f-aaab-0b740a174128 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:39.973328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:40.034599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7afa02-2019-4e6e-9b61-ab8f24c43cff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:40.035258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7afa02-2019-4e6e-9b61-ab8f24c43cff service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:40.035716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7afa02-2019-4e6e-9b61-ab8f24c43cff service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:40.035988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7afa02-2019-4e6e-9b61-ab8f24c43cff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:40.037800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8a7afa02-2019-4e6e-9b61-ab8f24c43cff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:40.131070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7afa02-2019-4e6e-9b61-ab8f24c43cff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:40.132708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7afa02-2019-4e6e-9b61-ab8f24c43cff service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:40.151726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 686/1376] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:42.360002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bd46a-f91d-4859-a8d6-bd0cf437532e None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:42.360488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bd46a-f91d-4859-a8d6-bd0cf437532e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:42.360602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bd46a-f91d-4859-a8d6-bd0cf437532e None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:42.361967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 691/1377] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:42.367379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90c1bf-8799-43b0-8866-1e4b9ce05e8e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:42.367841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90c1bf-8799-43b0-8866-1e4b9ce05e8e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:42.368207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90c1bf-8799-43b0-8866-1e4b9ce05e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:42.460500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6d90c1bf-8799-43b0-8866-1e4b9ce05e8e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:42.461730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6d90c1bf-8799-43b0-8866-1e4b9ce05e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:42.468554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:42.559159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 687/1378] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:42.604041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bd46a-f91d-4859-a8d6-bd0cf437532e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:42.605594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-df8bd46a-f91d-4859-a8d6-bd0cf437532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:42.674546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:42.781224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bd46a-f91d-4859-a8d6-bd0cf437532e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:42.930526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9978b-b11f-4c10-9070-0b05ca261600 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:42.931473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9978b-b11f-4c10-9070-0b05ca261600 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:42.931914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9978b-b11f-4c10-9070-0b05ca261600 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:42.932608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9978b-b11f-4c10-9070-0b05ca261600 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:42.934391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-13f9978b-b11f-4c10-9070-0b05ca261600 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:42.957945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:43.043950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9978b-b11f-4c10-9070-0b05ca261600 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:43.045960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9978b-b11f-4c10-9070-0b05ca261600 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:43.066800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 692/1379] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:45.270028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56544f1-f022-4c67-90b2-ed8dff6211f8 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:45.270588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56544f1-f022-4c67-90b2-ed8dff6211f8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:45.270588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56544f1-f022-4c67-90b2-ed8dff6211f8 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:45.272020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 688/1380] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:45.277271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a450f7f-97bb-4035-9f4a-10468f8dc14c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:45.277687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a450f7f-97bb-4035-9f4a-10468f8dc14c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:45.277998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a450f7f-97bb-4035-9f4a-10468f8dc14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:45.347149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7a450f7f-97bb-4035-9f4a-10468f8dc14c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:45.348405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7a450f7f-97bb-4035-9f4a-10468f8dc14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:45.447400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:45.454607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 693/1381] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:45.499698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56544f1-f022-4c67-90b2-ed8dff6211f8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:45.501757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e56544f1-f022-4c67-90b2-ed8dff6211f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:45.655118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:45.666268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56544f1-f022-4c67-90b2-ed8dff6211f8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:45.796719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed03c99-07e3-40c0-99ba-6a6a66e44e78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:45.797468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed03c99-07e3-40c0-99ba-6a6a66e44e78 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:45.797827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed03c99-07e3-40c0-99ba-6a6a66e44e78 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:45.798174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed03c99-07e3-40c0-99ba-6a6a66e44e78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:45.800350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3ed03c99-07e3-40c0-99ba-6a6a66e44e78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:45.921080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed03c99-07e3-40c0-99ba-6a6a66e44e78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:45.922965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed03c99-07e3-40c0-99ba-6a6a66e44e78 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:45.935135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:45.945900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 689/1382] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:48.616512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4454724-0453-4c84-9439-3e6b07fa9c45 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:48.617230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4454724-0453-4c84-9439-3e6b07fa9c45 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:48.617230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4454724-0453-4c84-9439-3e6b07fa9c45 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:48.618562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 694/1383] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:48.625013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a7d1f-5bcb-4936-a54a-4dbbf2f92102 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:48.625475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a7d1f-5bcb-4936-a54a-4dbbf2f92102 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:48.625854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a7d1f-5bcb-4936-a54a-4dbbf2f92102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:48.714960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f35a7d1f-5bcb-4936-a54a-4dbbf2f92102 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:48.716658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f35a7d1f-5bcb-4936-a54a-4dbbf2f92102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:48.839171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 690/1384] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:48.884053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4454724-0453-4c84-9439-3e6b07fa9c45 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:48.885584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e4454724-0453-4c84-9439-3e6b07fa9c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:49.043947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4454724-0453-4c84-9439-3e6b07fa9c45 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:49.119961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:49.175891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cc376-8351-4142-b77b-d50533daa3f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:49.176734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cc376-8351-4142-b77b-d50533daa3f8 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:49.177081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cc376-8351-4142-b77b-d50533daa3f8 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:49.177418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cc376-8351-4142-b77b-d50533daa3f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:49.179496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-489cc376-8351-4142-b77b-d50533daa3f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:49.288327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cc376-8351-4142-b77b-d50533daa3f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:49.294274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cc376-8351-4142-b77b-d50533daa3f8 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:49.316053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 695/1385] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:51.490542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22deec8-ee22-460d-9e67-1790568fbd66 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:51.491186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22deec8-ee22-460d-9e67-1790568fbd66 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:51.491186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22deec8-ee22-460d-9e67-1790568fbd66 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:51.492470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 691/1386] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:51.497868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f0c6e-7403-4435-aa3f-e7e107ee309e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:51.498287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f0c6e-7403-4435-aa3f-e7e107ee309e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:51.498460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f0c6e-7403-4435-aa3f-e7e107ee309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:51.571593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ce0f0c6e-7403-4435-aa3f-e7e107ee309e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:51.572890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ce0f0c6e-7403-4435-aa3f-e7e107ee309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:51.608822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:51.674122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 696/1387] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:51.720127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22deec8-ee22-460d-9e67-1790568fbd66 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:51.721752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f22deec8-ee22-460d-9e67-1790568fbd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:51.811231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:51.892143 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22deec8-ee22-460d-9e67-1790568fbd66 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:52.028999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd3350-5995-4517-bc9a-6db1fa908ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:52.029662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd3350-5995-4517-bc9a-6db1fa908ee3 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:52.030512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd3350-5995-4517-bc9a-6db1fa908ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:52.030797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd3350-5995-4517-bc9a-6db1fa908ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:52.032507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e6dd3350-5995-4517-bc9a-6db1fa908ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:52.075455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:52.133276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd3350-5995-4517-bc9a-6db1fa908ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:52.134954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd3350-5995-4517-bc9a-6db1fa908ee3 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:52.156936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 692/1388] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:54.390662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945cdaa-8162-4d57-bc0b-857c6e223d25 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:54.391215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945cdaa-8162-4d57-bc0b-857c6e223d25 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:54.391215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945cdaa-8162-4d57-bc0b-857c6e223d25 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:54.392671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 697/1389] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:54.398058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fb766-d820-461b-8941-ebd0d1f21be0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:54.398487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fb766-d820-461b-8941-ebd0d1f21be0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:54.398747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fb766-d820-461b-8941-ebd0d1f21be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:54.464191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3e0fb766-d820-461b-8941-ebd0d1f21be0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:54.465402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3e0fb766-d820-461b-8941-ebd0d1f21be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:54.562203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 693/1390] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:54.607063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945cdaa-8162-4d57-bc0b-857c6e223d25 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:54.608680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6945cdaa-8162-4d57-bc0b-857c6e223d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:54.627760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7dd4b-13f9-4e17-bbf3-7f4d85bdb21b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:54.628058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7dd4b-13f9-4e17-bbf3-7f4d85bdb21b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:54.628342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7dd4b-13f9-4e17-bbf3-7f4d85bdb21b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:54.629866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 694/1391] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:45:54.634684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250bf81-2b5f-404a-81fb-713315509aa2 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:54.634979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250bf81-2b5f-404a-81fb-713315509aa2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:54.635262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250bf81-2b5f-404a-81fb-713315509aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:54.650260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:54.701643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d250bf81-2b5f-404a-81fb-713315509aa2 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:54.703366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d250bf81-2b5f-404a-81fb-713315509aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:54.766322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945cdaa-8162-4d57-bc0b-857c6e223d25 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:54.785382 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:54.817597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 695/1392] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:45:54.878280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660a60f-973b-4dee-90b0-3cf860b16c41 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:54.878814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660a60f-973b-4dee-90b0-3cf860b16c41 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:54.879101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660a60f-973b-4dee-90b0-3cf860b16c41 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:54.880542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 696/1393] 10.222.0.98 () {54 vars in 960 bytes} [Fri Jul 15 15:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:45:54.888671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fcfcd-aa14-4177-95ef-27f15fc9405c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:54.888962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fcfcd-aa14-4177-95ef-27f15fc9405c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:54.890100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fcfcd-aa14-4177-95ef-27f15fc9405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:54.895312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:54.895969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:54.896255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:54.896530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:54.898159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:54.970165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2e4fcfcd-aa14-4177-95ef-27f15fc9405c None None] MFA Rules not processed for user `52f18d0f5f634d84a99fc3e46963a8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:54.971588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2e4fcfcd-aa14-4177-95ef-27f15fc9405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:54.995788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:54.997420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:55.011413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:55.017798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 698/1394] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:55.058084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 697/1395] 10.222.0.98 () {58 vars in 1057 bytes} [Fri Jul 15 15:45:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:55.066004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:55.067244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:55.090284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:55.146639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684f836-38c7-48d3-a0e6-a68547c405f5 service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:55.219745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a93fd1-8b45-477b-8246-993901032d88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f18d0f5f634d84a99fc3e46963a8b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:55.220450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93fd1-8b45-477b-8246-993901032d88 service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:55.220677 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93fd1-8b45-477b-8246-993901032d88 service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:55.220957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93fd1-8b45-477b-8246-993901032d88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:55.222594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d8a93fd1-8b45-477b-8246-993901032d88 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:55.338649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:55.353020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a93fd1-8b45-477b-8246-993901032d88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:55.354540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a93fd1-8b45-477b-8246-993901032d88 service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:55.380902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 699/1396] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:56.291797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87eb6b-f4b2-44b2-9155-ab1b65f7b1a1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:56.292257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87eb6b-f4b2-44b2-9155-ab1b65f7b1a1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:56.292364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87eb6b-f4b2-44b2-9155-ab1b65f7b1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:56.361623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7a87eb6b-f4b2-44b2-9155-ab1b65f7b1a1 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:56.362834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7a87eb6b-f4b2-44b2-9155-ab1b65f7b1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:56.457635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 698/1397] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:45:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:56.503853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac20ae9-ab49-447c-88eb-398d58f92c3b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:56.504322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac20ae9-ab49-447c-88eb-398d58f92c3b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:56.504486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac20ae9-ab49-447c-88eb-398d58f92c3b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:56.510533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 700/1398] 10.222.0.98 () {54 vars in 942 bytes} [Fri Jul 15 15:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:56.512226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8ed3-2b7a-4516-8011-8bf6d98a23e9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:56.512520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8ed3-2b7a-4516-8011-8bf6d98a23e9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:56.512797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8ed3-2b7a-4516-8011-8bf6d98a23e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:56.568680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:56.588476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-34df8ed3-2b7a-4516-8011-8bf6d98a23e9 None None] MFA Rules not processed for user `8f69205a5fc042299a8913c924b45e45`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:56.589686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-34df8ed3-2b7a-4516-8011-8bf6d98a23e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:56.697343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 699/1399] 10.222.0.98 () {58 vars in 1039 bytes} [Fri Jul 15 15:45:56 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:56.706764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac20ae9-ab49-447c-88eb-398d58f92c3b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:56.708381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5ac20ae9-ab49-447c-88eb-398d58f92c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:56.849582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac20ae9-ab49-447c-88eb-398d58f92c3b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:56.923203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:56.964808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd107b5f-ad72-49fa-9fc0-4502ca01bb7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:56.965654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd107b5f-ad72-49fa-9fc0-4502ca01bb7a service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:56.965988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd107b5f-ad72-49fa-9fc0-4502ca01bb7a service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:56.966484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd107b5f-ad72-49fa-9fc0-4502ca01bb7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:56.969072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd107b5f-ad72-49fa-9fc0-4502ca01bb7a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:57.052132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd107b5f-ad72-49fa-9fc0-4502ca01bb7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:57.053748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd107b5f-ad72-49fa-9fc0-4502ca01bb7a service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:57.062876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 701/1400] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:45:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:45:57.239807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c2cbb-7143-49a7-a405-9883443cf555 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:57.240124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c2cbb-7143-49a7-a405-9883443cf555 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:57.244724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c2cbb-7143-49a7-a405-9883443cf555 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:57.244724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 700/1401] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:45:57.247008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54727e-fa96-4bcc-be81-e9e914d76d5f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:57.247317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54727e-fa96-4bcc-be81-e9e914d76d5f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:57.247627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54727e-fa96-4bcc-be81-e9e914d76d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:57.313388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1b54727e-fa96-4bcc-be81-e9e914d76d5f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:45:57.314649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1b54727e-fa96-4bcc-be81-e9e914d76d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:57.368826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:57.416309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 702/1402] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:45:57.467958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c2cbb-7143-49a7-a405-9883443cf555 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:57.469483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7a5c2cbb-7143-49a7-a405-9883443cf555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:57.586834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:57.631499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c2cbb-7143-49a7-a405-9883443cf555 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:57.800322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aeaa5-8629-47bf-9b47-9ebe00bee673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:57.800899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aeaa5-8629-47bf-9b47-9ebe00bee673 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:57.801188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aeaa5-8629-47bf-9b47-9ebe00bee673 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:57.801524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aeaa5-8629-47bf-9b47-9ebe00bee673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:57.803491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bb4aeaa5-8629-47bf-9b47-9ebe00bee673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:57.897371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:57.922408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aeaa5-8629-47bf-9b47-9ebe00bee673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:57.924260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aeaa5-8629-47bf-9b47-9ebe00bee673 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:57.947386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 701/1403] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:00.199201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f3f9-4233-4fed-98e1-42cb006d78db None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:00.199848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f3f9-4233-4fed-98e1-42cb006d78db None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:00.200303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f3f9-4233-4fed-98e1-42cb006d78db None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:00.207484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 703/1404] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:46:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:00.210594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15281f0-900f-449c-a49f-9ea5824ddf6a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:00.211078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15281f0-900f-449c-a49f-9ea5824ddf6a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:00.211179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15281f0-900f-449c-a49f-9ea5824ddf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:00.287651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c15281f0-900f-449c-a49f-9ea5824ddf6a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:00.288970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c15281f0-900f-449c-a49f-9ea5824ddf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:00.406969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 702/1405] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:00.453818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193f3f9-4233-4fed-98e1-42cb006d78db None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:00.455374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8193f3f9-4233-4fed-98e1-42cb006d78db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:00.618064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193f3f9-4233-4fed-98e1-42cb006d78db None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:00.661248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:00.743282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880670fd-e678-4f76-ac62-0af11a18305b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:00.744079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880670fd-e678-4f76-ac62-0af11a18305b service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:00.744371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880670fd-e678-4f76-ac62-0af11a18305b service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:00.744647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880670fd-e678-4f76-ac62-0af11a18305b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:00.746382 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-880670fd-e678-4f76-ac62-0af11a18305b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:00.842205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880670fd-e678-4f76-ac62-0af11a18305b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:00.843917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880670fd-e678-4f76-ac62-0af11a18305b service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:00.863470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 704/1406] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:03.125279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733548-4e8a-4b70-8a62-1fcb8c8b302f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:03.125845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733548-4e8a-4b70-8a62-1fcb8c8b302f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:03.126199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733548-4e8a-4b70-8a62-1fcb8c8b302f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:03.127716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 703/1407] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:03.133287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b002-c1e5-41d1-95af-0da5763d6582 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:03.133769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b002-c1e5-41d1-95af-0da5763d6582 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:03.133852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b002-c1e5-41d1-95af-0da5763d6582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:03.189797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:03.204974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4263b002-c1e5-41d1-95af-0da5763d6582 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:03.206353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4263b002-c1e5-41d1-95af-0da5763d6582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:03.324723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 705/1408] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:03.372161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76733548-4e8a-4b70-8a62-1fcb8c8b302f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:03.374511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-76733548-4e8a-4b70-8a62-1fcb8c8b302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:03.432355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:03.541225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76733548-4e8a-4b70-8a62-1fcb8c8b302f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:03.669419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0776d-f8a0-44c2-aaa0-4ec96e78fedb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:03.670215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0776d-f8a0-44c2-aaa0-4ec96e78fedb service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:03.670513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0776d-f8a0-44c2-aaa0-4ec96e78fedb service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:03.671459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0776d-f8a0-44c2-aaa0-4ec96e78fedb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:03.673124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-29a0776d-f8a0-44c2-aaa0-4ec96e78fedb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:03.682544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:03.776707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0776d-f8a0-44c2-aaa0-4ec96e78fedb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:03.778343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0776d-f8a0-44c2-aaa0-4ec96e78fedb service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:03.798197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 704/1409] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:13.322834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=46 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:13.795448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=41 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:22.008341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ac31-f618-4893-a202-c3619ffbf232 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:22.008832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ac31-f618-4893-a202-c3619ffbf232 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:22.008963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ac31-f618-4893-a202-c3619ffbf232 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:22.010928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 706/1410] 10.222.0.98 () {54 vars in 918 bytes} [Fri Jul 15 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:22.016085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa847d77-aff1-4437-ae7a-700c99af470e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:22.017826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa847d77-aff1-4437-ae7a-700c99af470e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:22.017826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa847d77-aff1-4437-ae7a-700c99af470e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:22.094046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fa847d77-aff1-4437-ae7a-700c99af470e None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:22.095321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fa847d77-aff1-4437-ae7a-700c99af470e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:22.200955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 705/1411] 10.222.0.98 () {58 vars in 1015 bytes} [Fri Jul 15 15:46:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:22.215424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf667-d2e2-4c3a-9081-83343718a2f1 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:22.215722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf667-d2e2-4c3a-9081-83343718a2f1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:22.216007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf667-d2e2-4c3a-9081-83343718a2f1 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:22.217387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 707/1412] 10.222.0.98 () {54 vars in 918 bytes} [Fri Jul 15 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:22.223343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c5a42-0e93-4c31-9f8d-4d86771bccb8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:22.223652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c5a42-0e93-4c31-9f8d-4d86771bccb8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:22.223970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c5a42-0e93-4c31-9f8d-4d86771bccb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:22.296554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:22.299753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e21c5a42-0e93-4c31-9f8d-4d86771bccb8 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:22.300986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e21c5a42-0e93-4c31-9f8d-4d86771bccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:22.417658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 706/1413] 10.222.0.98 () {58 vars in 1015 bytes} [Fri Jul 15 15:46:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:24.901066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e53399-be06-433a-addd-fed9d4e14cec None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:24.901642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e53399-be06-433a-addd-fed9d4e14cec None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:24.901642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e53399-be06-433a-addd-fed9d4e14cec None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:24.903018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:24.908748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde25428-2c13-43e0-a634-9ab7cc188cf4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:24.909130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde25428-2c13-43e0-a634-9ab7cc188cf4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:24.909315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde25428-2c13-43e0-a634-9ab7cc188cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:24.974228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-bde25428-2c13-43e0-a634-9ab7cc188cf4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:24.975471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bde25428-2c13-43e0-a634-9ab7cc188cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:25.030219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:25.072492 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:25.116099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e53399-be06-433a-addd-fed9d4e14cec None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:25.117633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b0e53399-be06-433a-addd-fed9d4e14cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:25.262823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e53399-be06-433a-addd-fed9d4e14cec None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:25.342429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:25.383521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2f7a2-1253-4693-9f61-3370fff071d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:25.384186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f7a2-1253-4693-9f61-3370fff071d0 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:25.384481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f7a2-1253-4693-9f61-3370fff071d0 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:25.384768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f7a2-1253-4693-9f61-3370fff071d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:25.386426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4bc2f7a2-1253-4693-9f61-3370fff071d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:25.475673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2f7a2-1253-4693-9f61-3370fff071d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:25.477274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2f7a2-1253-4693-9f61-3370fff071d0 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:25.495021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 709/1416] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:26.398441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb66d-4c52-4d32-a165-5df7557c9c39 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:26.398996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb66d-4c52-4d32-a165-5df7557c9c39 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:26.398996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb66d-4c52-4d32-a165-5df7557c9c39 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:26.400441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:26.405996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71456c43-427f-4301-a4a3-3ec01944536c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:26.406405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71456c43-427f-4301-a4a3-3ec01944536c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:26.406593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71456c43-427f-4301-a4a3-3ec01944536c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:26.481988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-71456c43-427f-4301-a4a3-3ec01944536c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:26.483212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-71456c43-427f-4301-a4a3-3ec01944536c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:26.510313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:26.579069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:26.728803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eb66d-4c52-4d32-a165-5df7557c9c39 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:26.730330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b59eb66d-4c52-4d32-a165-5df7557c9c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:26.876021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eb66d-4c52-4d32-a165-5df7557c9c39 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:26.916184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:26.989013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba35ac-f664-4f14-b1d1-33bf8b4e0555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:26.989585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba35ac-f664-4f14-b1d1-33bf8b4e0555 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:26.989873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba35ac-f664-4f14-b1d1-33bf8b4e0555 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:26.990153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba35ac-f664-4f14-b1d1-33bf8b4e0555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:26.991843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-32ba35ac-f664-4f14-b1d1-33bf8b4e0555 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:27.081644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba35ac-f664-4f14-b1d1-33bf8b4e0555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:27.083270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba35ac-f664-4f14-b1d1-33bf8b4e0555 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:27.101840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 709/1419] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:27.993940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c4c3-fdab-45cb-b268-3ae78cdc27de None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:27.994498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c4c3-fdab-45cb-b268-3ae78cdc27de None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:27.994498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c4c3-fdab-45cb-b268-3ae78cdc27de None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:27.995923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:28.001424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e984-c9e8-4bd2-9f3e-c06294ec5c5d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:28.001874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e984-c9e8-4bd2-9f3e-c06294ec5c5d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:28.001981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e984-c9e8-4bd2-9f3e-c06294ec5c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:28.055884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:28.077314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b955e984-c9e8-4bd2-9f3e-c06294ec5c5d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:28.078566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b955e984-c9e8-4bd2-9f3e-c06294ec5c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:28.176084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:28.326038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c4c3-fdab-45cb-b268-3ae78cdc27de None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:28.328026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7d48c4c3-fdab-45cb-b268-3ae78cdc27de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:28.476556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c4c3-fdab-45cb-b268-3ae78cdc27de None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:28.488817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:28.589578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d948ef6-863b-49d3-ae70-67f1e36ea659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:28.590130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d948ef6-863b-49d3-ae70-67f1e36ea659 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:28.590441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d948ef6-863b-49d3-ae70-67f1e36ea659 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:28.590694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d948ef6-863b-49d3-ae70-67f1e36ea659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:28.592374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7d948ef6-863b-49d3-ae70-67f1e36ea659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:28.680707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d948ef6-863b-49d3-ae70-67f1e36ea659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:28.682302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d948ef6-863b-49d3-ae70-67f1e36ea659 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:28.700924 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 712/1422] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:29.570541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511aedb-85f7-4675-9dfe-1f701526079a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:29.571100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511aedb-85f7-4675-9dfe-1f701526079a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:29.571100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511aedb-85f7-4675-9dfe-1f701526079a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:29.572526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:29.577379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474677d-048e-4e7e-a40d-7e43c009ee63 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:29.577778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474677d-048e-4e7e-a40d-7e43c009ee63 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:29.577973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474677d-048e-4e7e-a40d-7e43c009ee63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:29.596554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:29.651223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f474677d-048e-4e7e-a40d-7e43c009ee63 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:29.652457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f474677d-048e-4e7e-a40d-7e43c009ee63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:29.778518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:29.931406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511aedb-85f7-4675-9dfe-1f701526079a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:29.932857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b511aedb-85f7-4675-9dfe-1f701526079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:30.045465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:30.082396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511aedb-85f7-4675-9dfe-1f701526079a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:30.201792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a916170-9570-476a-9695-478a705166fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:30.202356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a916170-9570-476a-9695-478a705166fb service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:30.203208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a916170-9570-476a-9695-478a705166fb service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:30.203512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a916170-9570-476a-9695-478a705166fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:30.205150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0a916170-9570-476a-9695-478a705166fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:30.279250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:30.292811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a916170-9570-476a-9695-478a705166fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:30.294442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a916170-9570-476a-9695-478a705166fb service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:30.314863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 712/1425] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:46:31.173853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0116-91af-4bde-bb71-0c341d519733 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:31.174422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0116-91af-4bde-bb71-0c341d519733 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:31.174422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0116-91af-4bde-bb71-0c341d519733 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:31.175852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:31.180792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7f52c-f6c4-428c-8c83-1b58a914387c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:31.181192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7f52c-f6c4-428c-8c83-1b58a914387c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:31.181417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7f52c-f6c4-428c-8c83-1b58a914387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:31.256500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-adf7f52c-f6c4-428c-8c83-1b58a914387c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:31.258014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-adf7f52c-f6c4-428c-8c83-1b58a914387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:31.358749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:31.508496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf0116-91af-4bde-bb71-0c341d519733 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:31.510034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7faf0116-91af-4bde-bb71-0c341d519733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:31.594582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:31.654698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf0116-91af-4bde-bb71-0c341d519733 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:31.764917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af518508-9ca7-42c8-9e61-d841a8dd94fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:31.766044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af518508-9ca7-42c8-9e61-d841a8dd94fa service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:31.766328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af518508-9ca7-42c8-9e61-d841a8dd94fa service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:31.766603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af518508-9ca7-42c8-9e61-d841a8dd94fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:31.770899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-af518508-9ca7-42c8-9e61-d841a8dd94fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:31.820481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:31.861453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af518508-9ca7-42c8-9e61-d841a8dd94fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:31.863077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af518508-9ca7-42c8-9e61-d841a8dd94fa service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:31.881111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 715/1428] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:32.744300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d3e6f-84f6-4775-9c85-8d64d23a4c1a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:32.744840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d3e6f-84f6-4775-9c85-8d64d23a4c1a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:32.744940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d3e6f-84f6-4775-9c85-8d64d23a4c1a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:32.746273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:32.751217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344b34e-6310-4f2d-ba7c-abd7f39d6296 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:32.751764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344b34e-6310-4f2d-ba7c-abd7f39d6296 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:32.751876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344b34e-6310-4f2d-ba7c-abd7f39d6296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:32.829980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4344b34e-6310-4f2d-ba7c-abd7f39d6296 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:32.831209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4344b34e-6310-4f2d-ba7c-abd7f39d6296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:32.925688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:33.074073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d3e6f-84f6-4775-9c85-8d64d23a4c1a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:33.075553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cc3d3e6f-84f6-4775-9c85-8d64d23a4c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:33.129248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:33.229525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d3e6f-84f6-4775-9c85-8d64d23a4c1a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:33.343675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ebea2-90fa-4c4b-b334-4c8ad7428211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:33.344280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ebea2-90fa-4c4b-b334-4c8ad7428211 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:33.344562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ebea2-90fa-4c4b-b334-4c8ad7428211 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:33.344838 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ebea2-90fa-4c4b-b334-4c8ad7428211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:33.346464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-883ebea2-90fa-4c4b-b334-4c8ad7428211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:33.353682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:33.445422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ebea2-90fa-4c4b-b334-4c8ad7428211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:33.447084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ebea2-90fa-4c4b-b334-4c8ad7428211 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:33.467551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 715/1431] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:34.397072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c705-6d26-4ad4-a83e-d8d415be3b44 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:34.397633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c705-6d26-4ad4-a83e-d8d415be3b44 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:34.397753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c705-6d26-4ad4-a83e-d8d415be3b44 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:34.399077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:46:34.404008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129f243-412b-4ead-95d7-73728e4e19d5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:34.404417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129f243-412b-4ead-95d7-73728e4e19d5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:34.404586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129f243-412b-4ead-95d7-73728e4e19d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:34.478881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0129f243-412b-4ead-95d7-73728e4e19d5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:34.480135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0129f243-412b-4ead-95d7-73728e4e19d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:34.546945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:34.577096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:34.729804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c705-6d26-4ad4-a83e-d8d415be3b44 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:34.731351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6f64c705-6d26-4ad4-a83e-d8d415be3b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:34.749280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:34.880318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c705-6d26-4ad4-a83e-d8d415be3b44 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:34.965865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:34.991603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc4628-4efd-4ec4-b73e-598b28e1a689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:34.992251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4628-4efd-4ec4-b73e-598b28e1a689 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:34.992534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4628-4efd-4ec4-b73e-598b28e1a689 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:34.992805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4628-4efd-4ec4-b73e-598b28e1a689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:34.994414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-60cc4628-4efd-4ec4-b73e-598b28e1a689 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:35.082013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4628-4efd-4ec4-b73e-598b28e1a689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:35.083810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4628-4efd-4ec4-b73e-598b28e1a689 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:35.102546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 718/1434] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:35.980129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4914e-dd38-41a6-924d-7c51214fbde0 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:35.980700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4914e-dd38-41a6-924d-7c51214fbde0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:35.980800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4914e-dd38-41a6-924d-7c51214fbde0 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:35.982170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:35.987717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920e39d-39fb-438b-8131-4219e9990f6f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:35.988158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920e39d-39fb-438b-8131-4219e9990f6f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:35.988291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920e39d-39fb-438b-8131-4219e9990f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:36.062639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0920e39d-39fb-438b-8131-4219e9990f6f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:36.063885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0920e39d-39fb-438b-8131-4219e9990f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:36.103750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:36.158195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:36.308734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4914e-dd38-41a6-924d-7c51214fbde0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:36.310442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-67c4914e-dd38-41a6-924d-7c51214fbde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:36.456204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4914e-dd38-41a6-924d-7c51214fbde0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:36.515213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:36.580685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c520e0-2a8b-42a6-ad4d-8b72e2d081e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:36.581290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c520e0-2a8b-42a6-ad4d-8b72e2d081e7 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:36.581581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c520e0-2a8b-42a6-ad4d-8b72e2d081e7 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:36.581839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c520e0-2a8b-42a6-ad4d-8b72e2d081e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:36.583511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d5c520e0-2a8b-42a6-ad4d-8b72e2d081e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:36.677927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c520e0-2a8b-42a6-ad4d-8b72e2d081e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:36.727589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c520e0-2a8b-42a6-ad4d-8b72e2d081e7 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:36.727589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 718/1437] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:37.660559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ad49a-cc15-40a6-911b-69e7e433c202 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:37.661127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ad49a-cc15-40a6-911b-69e7e433c202 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:37.661127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ad49a-cc15-40a6-911b-69e7e433c202 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:37.662456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:37.667393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16aa5-f68b-4ce9-a771-7924b3bd09b7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:37.667827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16aa5-f68b-4ce9-a771-7924b3bd09b7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:37.667958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16aa5-f68b-4ce9-a771-7924b3bd09b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:37.740619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-28c16aa5-f68b-4ce9-a771-7924b3bd09b7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:37.741817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-28c16aa5-f68b-4ce9-a771-7924b3bd09b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:37.743875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:37.841095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:38.000573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ad49a-cc15-40a6-911b-69e7e433c202 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:38.002130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-343ad49a-cc15-40a6-911b-69e7e433c202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:38.147497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ad49a-cc15-40a6-911b-69e7e433c202 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:38.174334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:38.259589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf7529-8a4b-4e45-8bb8-a3d03580ecde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:38.260166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf7529-8a4b-4e45-8bb8-a3d03580ecde service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:38.260451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf7529-8a4b-4e45-8bb8-a3d03580ecde service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:38.260725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf7529-8a4b-4e45-8bb8-a3d03580ecde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:38.262353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7baf7529-8a4b-4e45-8bb8-a3d03580ecde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:38.350273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf7529-8a4b-4e45-8bb8-a3d03580ecde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:38.351895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf7529-8a4b-4e45-8bb8-a3d03580ecde service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:38.369755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 721/1440] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:40.302248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbcd6-73f5-40ed-9590-111d56ace600 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:40.302769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbcd6-73f5-40ed-9590-111d56ace600 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:40.302769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebbcd6-73f5-40ed-9590-111d56ace600 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:40.304157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:40.309561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cb86f-854b-462f-bbd3-19c59440cc88 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:40.310007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cb86f-854b-462f-bbd3-19c59440cc88 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:40.310111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cb86f-854b-462f-bbd3-19c59440cc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:40.351192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:40.375410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7b4cb86f-854b-462f-bbd3-19c59440cc88 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:40.376637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7b4cb86f-854b-462f-bbd3-19c59440cc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:40.479049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:40.517527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbcd6-73f5-40ed-9590-111d56ace600 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:40.519055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-39ebbcd6-73f5-40ed-9590-111d56ace600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:40.623362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebbcd6-73f5-40ed-9590-111d56ace600 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:40.655442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:40.698898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faf159-6f88-4263-8514-af352f219d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:40.699586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faf159-6f88-4263-8514-af352f219d12 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:40.699871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faf159-6f88-4263-8514-af352f219d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:40.700148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faf159-6f88-4263-8514-af352f219d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:40.701789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f7faf159-6f88-4263-8514-af352f219d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:40.808701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faf159-6f88-4263-8514-af352f219d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:40.810156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faf159-6f88-4263-8514-af352f219d12 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:40.829098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 721/1443] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:42.987285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73027576-518c-4083-b4ca-b6cf7d0ad209 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:42.987893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73027576-518c-4083-b4ca-b6cf7d0ad209 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:42.987992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73027576-518c-4083-b4ca-b6cf7d0ad209 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:42.989384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:42.994680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0249321-5b14-45f5-9fec-88ef87c6f08a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:42.995114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0249321-5b14-45f5-9fec-88ef87c6f08a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:42.995219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0249321-5b14-45f5-9fec-88ef87c6f08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:43.069841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c0249321-5b14-45f5-9fec-88ef87c6f08a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:43.071041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c0249321-5b14-45f5-9fec-88ef87c6f08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:43.084172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:43.166450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:43.204904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73027576-518c-4083-b4ca-b6cf7d0ad209 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:43.206474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-73027576-518c-4083-b4ca-b6cf7d0ad209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:43.308027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73027576-518c-4083-b4ca-b6cf7d0ad209 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:43.326310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:43.388747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3052cc-5c6a-442f-b7c4-106b45a4c778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:43.389403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3052cc-5c6a-442f-b7c4-106b45a4c778 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:43.389688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3052cc-5c6a-442f-b7c4-106b45a4c778 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:43.389963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3052cc-5c6a-442f-b7c4-106b45a4c778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:43.391656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9f3052cc-5c6a-442f-b7c4-106b45a4c778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:43.501094 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3052cc-5c6a-442f-b7c4-106b45a4c778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:43.502561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3052cc-5c6a-442f-b7c4-106b45a4c778 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:43.520155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 724/1446] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:45.932854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217e856-d2f7-4177-8247-86594d095913 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:45.933438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217e856-d2f7-4177-8247-86594d095913 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:45.933438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217e856-d2f7-4177-8247-86594d095913 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:45.934825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:45.940833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6f74-d210-474c-ad8e-80e2a6a4e7c7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:45.941253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6f74-d210-474c-ad8e-80e2a6a4e7c7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:45.941396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6f74-d210-474c-ad8e-80e2a6a4e7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:45.988481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:46.006393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-955f6f74-d210-474c-ad8e-80e2a6a4e7c7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:46.007774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-955f6f74-d210-474c-ad8e-80e2a6a4e7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:46.110245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:46.148581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217e856-d2f7-4177-8247-86594d095913 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:46.150114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0217e856-d2f7-4177-8247-86594d095913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:46.254788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217e856-d2f7-4177-8247-86594d095913 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:46.294673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:46.329392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82baa0c-c8d0-4ee4-9915-1eaf242ef02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:46.330043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82baa0c-c8d0-4ee4-9915-1eaf242ef02f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:46.330329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82baa0c-c8d0-4ee4-9915-1eaf242ef02f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:46.330610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82baa0c-c8d0-4ee4-9915-1eaf242ef02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:46.332281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b82baa0c-c8d0-4ee4-9915-1eaf242ef02f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:46.441862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82baa0c-c8d0-4ee4-9915-1eaf242ef02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:46.443359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82baa0c-c8d0-4ee4-9915-1eaf242ef02f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:46.462058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 724/1449] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:48.599456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b4a0-d2ea-4206-a7b3-27674726363d None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:48.600005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b4a0-d2ea-4206-a7b3-27674726363d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:48.600005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201b4a0-d2ea-4206-a7b3-27674726363d None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:48.601389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:48.606686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5dc1-47c1-4f5b-857a-cfb540fe0af0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:48.607086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5dc1-47c1-4f5b-857a-cfb540fe0af0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:48.607260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb5dc1-47c1-4f5b-857a-cfb540fe0af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:48.671322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a9fb5dc1-47c1-4f5b-857a-cfb540fe0af0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:48.672967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a9fb5dc1-47c1-4f5b-857a-cfb540fe0af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:48.698864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:48.792094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:48.832130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201b4a0-d2ea-4206-a7b3-27674726363d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:48.833653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f201b4a0-d2ea-4206-a7b3-27674726363d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:48.935429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201b4a0-d2ea-4206-a7b3-27674726363d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:48.955313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:49.008422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3b26e-a743-4ebf-bc40-477fc3dd421a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:49.009080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b26e-a743-4ebf-bc40-477fc3dd421a service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:49.009378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b26e-a743-4ebf-bc40-477fc3dd421a service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:49.009656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3b26e-a743-4ebf-bc40-477fc3dd421a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:49.011368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-07e3b26e-a743-4ebf-bc40-477fc3dd421a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:49.121996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3b26e-a743-4ebf-bc40-477fc3dd421a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:49.123471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3b26e-a743-4ebf-bc40-477fc3dd421a service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:49.141242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 727/1452] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:51.411567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1045f-f574-4b4f-8b02-7e10066dd71d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:51.412128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1045f-f574-4b4f-8b02-7e10066dd71d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:51.412656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1045f-f574-4b4f-8b02-7e10066dd71d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:51.414682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:51.420301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e67a7-a117-474e-869a-3429c6d83a15 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:51.420718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e67a7-a117-474e-869a-3429c6d83a15 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:51.420866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e67a7-a117-474e-869a-3429c6d83a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:51.487551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:51.490180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-476e67a7-a117-474e-869a-3429c6d83a15 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:51.491427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-476e67a7-a117-474e-869a-3429c6d83a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:51.592583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:51.631304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1045f-f574-4b4f-8b02-7e10066dd71d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:51.632871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2a1045f-f574-4b4f-8b02-7e10066dd71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:51.736486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1045f-f574-4b4f-8b02-7e10066dd71d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:51.785699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:51.809836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4eaf86-0d58-461b-9580-503d909c0d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:51.810483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4eaf86-0d58-461b-9580-503d909c0d99 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:51.810769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4eaf86-0d58-461b-9580-503d909c0d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:51.811348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4eaf86-0d58-461b-9580-503d909c0d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:51.813013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2f4eaf86-0d58-461b-9580-503d909c0d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:51.920998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4eaf86-0d58-461b-9580-503d909c0d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:51.922452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4eaf86-0d58-461b-9580-503d909c0d99 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:51.941540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 727/1455] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:54.049631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b664f4e-3baa-4c4f-90e1-652ed17a4c49 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:54.050187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b664f4e-3baa-4c4f-90e1-652ed17a4c49 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:54.050315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b664f4e-3baa-4c4f-90e1-652ed17a4c49 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:54.052137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:54.057575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6797f07-8acc-4526-9abf-b06c49cdf4ac None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:54.058038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6797f07-8acc-4526-9abf-b06c49cdf4ac None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:54.058146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6797f07-8acc-4526-9abf-b06c49cdf4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:54.122534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d6797f07-8acc-4526-9abf-b06c49cdf4ac None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:54.123771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d6797f07-8acc-4526-9abf-b06c49cdf4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:54.152495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:54.223480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:46:54.261547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b664f4e-3baa-4c4f-90e1-652ed17a4c49 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:54.263113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5b664f4e-3baa-4c4f-90e1-652ed17a4c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:54.367689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b664f4e-3baa-4c4f-90e1-652ed17a4c49 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:54.399313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:54.450835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27247830-8ac1-4e10-8aab-66337c119a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:54.451515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247830-8ac1-4e10-8aab-66337c119a57 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:54.451810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247830-8ac1-4e10-8aab-66337c119a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:54.452087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27247830-8ac1-4e10-8aab-66337c119a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:54.454317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-27247830-8ac1-4e10-8aab-66337c119a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:54.565101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27247830-8ac1-4e10-8aab-66337c119a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:54.566663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27247830-8ac1-4e10-8aab-66337c119a57 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:54.585258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 730/1458] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:56.657854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7213d-c781-4b96-899a-b8dd4de65110 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:56.658489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7213d-c781-4b96-899a-b8dd4de65110 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:56.658489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7213d-c781-4b96-899a-b8dd4de65110 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:56.659890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:56.665098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ce74-51f9-4a55-b8a8-554bb280612f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:56.665474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ce74-51f9-4a55-b8a8-554bb280612f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:56.665670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ce74-51f9-4a55-b8a8-554bb280612f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:56.729550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d024ce74-51f9-4a55-b8a8-554bb280612f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:56.730748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d024ce74-51f9-4a55-b8a8-554bb280612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:56.738881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:56.826643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:56.863817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7213d-c781-4b96-899a-b8dd4de65110 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:56.865363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-90b7213d-c781-4b96-899a-b8dd4de65110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:56.968316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7213d-c781-4b96-899a-b8dd4de65110 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:57.024652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:57.042629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66933ad6-a7cb-4174-9335-c87ff0a7ff63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:57.058765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933ad6-a7cb-4174-9335-c87ff0a7ff63 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:57.058765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933ad6-a7cb-4174-9335-c87ff0a7ff63 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:57.058765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66933ad6-a7cb-4174-9335-c87ff0a7ff63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:57.058765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-66933ad6-a7cb-4174-9335-c87ff0a7ff63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:57.154540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66933ad6-a7cb-4174-9335-c87ff0a7ff63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:57.156127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66933ad6-a7cb-4174-9335-c87ff0a7ff63 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:57.175023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 730/1461] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:59.292594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa672d5-e443-4356-9579-77a7b1421d46 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:59.293193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa672d5-e443-4356-9579-77a7b1421d46 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:59.293193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa672d5-e443-4356-9579-77a7b1421d46 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:59.294575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:46:59.300324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739573c-ffc2-4dd5-be44-41ede2c64970 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:59.300925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739573c-ffc2-4dd5-be44-41ede2c64970 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:59.300925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739573c-ffc2-4dd5-be44-41ede2c64970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:59.366274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6739573c-ffc2-4dd5-be44-41ede2c64970 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:59.367494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6739573c-ffc2-4dd5-be44-41ede2c64970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:59.405595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:59.463946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:59.502681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa672d5-e443-4356-9579-77a7b1421d46 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:59.504268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ffa672d5-e443-4356-9579-77a7b1421d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:59.610639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa672d5-e443-4356-9579-77a7b1421d46 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:59.647496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:59.685012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0a27a-db8b-48a2-9fbe-07f5d9b85ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:59.685666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a27a-db8b-48a2-9fbe-07f5d9b85ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:59.685948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a27a-db8b-48a2-9fbe-07f5d9b85ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:59.686223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a27a-db8b-48a2-9fbe-07f5d9b85ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:59.688179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-beb0a27a-db8b-48a2-9fbe-07f5d9b85ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:59.796997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0a27a-db8b-48a2-9fbe-07f5d9b85ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:59.798588 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0a27a-db8b-48a2-9fbe-07f5d9b85ca4 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:59.818574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 733/1464] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:01.880070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41fe6b-af5a-462d-b1c2-e1c5c0d41d1f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:01.880608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41fe6b-af5a-462d-b1c2-e1c5c0d41d1f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:01.880721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41fe6b-af5a-462d-b1c2-e1c5c0d41d1f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:01.882029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:01.887435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da6d8-67be-4a75-ae43-fff3fe89935e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:01.887875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da6d8-67be-4a75-ae43-fff3fe89935e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:01.888005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da6d8-67be-4a75-ae43-fff3fe89935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:01.956261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-312da6d8-67be-4a75-ae43-fff3fe89935e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:01.957481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-312da6d8-67be-4a75-ae43-fff3fe89935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:01.975360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:02.055516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:02.092911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41fe6b-af5a-462d-b1c2-e1c5c0d41d1f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:02.094514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fe41fe6b-af5a-462d-b1c2-e1c5c0d41d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:02.198930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41fe6b-af5a-462d-b1c2-e1c5c0d41d1f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:02.263734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:02.270284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf6219-c2de-4828-8267-3b1cc29ed0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:02.270928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf6219-c2de-4828-8267-3b1cc29ed0fb service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:02.271216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf6219-c2de-4828-8267-3b1cc29ed0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:02.271499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf6219-c2de-4828-8267-3b1cc29ed0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:02.273122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-81bf6219-c2de-4828-8267-3b1cc29ed0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:02.381146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf6219-c2de-4828-8267-3b1cc29ed0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:02.382603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf6219-c2de-4828-8267-3b1cc29ed0fb service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:02.401381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 733/1467] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:04.418173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f672-8be6-4d81-8f5d-c87ae2be3cf2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:04.418578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f672-8be6-4d81-8f5d-c87ae2be3cf2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:04.418741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f672-8be6-4d81-8f5d-c87ae2be3cf2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:04.420254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:04.426028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63698f5e-f1c7-4650-be18-dec6414eb289 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:04.426452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63698f5e-f1c7-4650-be18-dec6414eb289 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:04.426587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63698f5e-f1c7-4650-be18-dec6414eb289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:04.490728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-63698f5e-f1c7-4650-be18-dec6414eb289 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:04.491972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-63698f5e-f1c7-4650-be18-dec6414eb289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:04.540257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:04.588875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:04.628466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008f672-8be6-4d81-8f5d-c87ae2be3cf2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:04.630383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4008f672-8be6-4d81-8f5d-c87ae2be3cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:04.733368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008f672-8be6-4d81-8f5d-c87ae2be3cf2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:04.780680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:04.807597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffc73c-02cf-4ead-86e2-45679f3f4a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:04.808280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc73c-02cf-4ead-86e2-45679f3f4a69 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:04.808564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc73c-02cf-4ead-86e2-45679f3f4a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:04.808843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc73c-02cf-4ead-86e2-45679f3f4a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:04.811277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a2ffc73c-02cf-4ead-86e2-45679f3f4a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:04.918250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffc73c-02cf-4ead-86e2-45679f3f4a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:04.922734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffc73c-02cf-4ead-86e2-45679f3f4a69 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:04.938938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 736/1470] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:47:14.586994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=22 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:14.936288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=66 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:24.526994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9aee1-2080-416e-bfd9-fe11a9b8ee0e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:24.527500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9aee1-2080-416e-bfd9-fe11a9b8ee0e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:24.527593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9aee1-2080-416e-bfd9-fe11a9b8ee0e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:24.528991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 735/1471] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:24.534369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be87c6-e18e-4027-8051-1baa1a0d6e31 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:24.534814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be87c6-e18e-4027-8051-1baa1a0d6e31 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:24.534919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be87c6-e18e-4027-8051-1baa1a0d6e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:24.599251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-10be87c6-e18e-4027-8051-1baa1a0d6e31 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:24.600487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-10be87c6-e18e-4027-8051-1baa1a0d6e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:24.697583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 737/1472] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:24.745672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1ff50-d564-48c3-9d19-98516bd485d1 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:24.745991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1ff50-d564-48c3-9d19-98516bd485d1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:24.746275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1ff50-d564-48c3-9d19-98516bd485d1 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:24.747728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 736/1473] 10.222.0.98 () {54 vars in 960 bytes} [Fri Jul 15 15:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:24.753085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd8af2-df65-43ff-8b17-8f65cdbe2228 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:24.753383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd8af2-df65-43ff-8b17-8f65cdbe2228 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:24.753666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd8af2-df65-43ff-8b17-8f65cdbe2228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:24.826773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3afd8af2-df65-43ff-8b17-8f65cdbe2228 None None] MFA Rules not processed for user `52f18d0f5f634d84a99fc3e46963a8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:24.828011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3afd8af2-df65-43ff-8b17-8f65cdbe2228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:24.848155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:24.904889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 738/1474] 10.222.0.98 () {58 vars in 1057 bytes} [Fri Jul 15 15:47:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:24.912957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1ff50-d564-48c3-9d19-98516bd485d1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:24.914557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-70b1ff50-d564-48c3-9d19-98516bd485d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:25.018323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1ff50-d564-48c3-9d19-98516bd485d1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:25.091498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f689d4a-2b77-4544-991e-e9e24dbd3338 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f18d0f5f634d84a99fc3e46963a8b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:25.092154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689d4a-2b77-4544-991e-e9e24dbd3338 service glance] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:25.092443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689d4a-2b77-4544-991e-e9e24dbd3338 service glance] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:25.092725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689d4a-2b77-4544-991e-e9e24dbd3338 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:25.094868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8f689d4a-2b77-4544-991e-e9e24dbd3338 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:25.147729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:25.202032 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f689d4a-2b77-4544-991e-e9e24dbd3338 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:25.203508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f689d4a-2b77-4544-991e-e9e24dbd3338 service glance] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:25.223342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 737/1475] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:27.356802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc44ef-7981-4f81-87aa-80c01a26f812 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:27.357375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc44ef-7981-4f81-87aa-80c01a26f812 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:27.357375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc44ef-7981-4f81-87aa-80c01a26f812 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:27.358767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 739/1476] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:27.363982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6e4c-3722-4f0a-8983-f3869f930378 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:27.364418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6e4c-3722-4f0a-8983-f3869f930378 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:27.364577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a6e4c-3722-4f0a-8983-f3869f930378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:27.428660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-588a6e4c-3722-4f0a-8983-f3869f930378 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:27.429870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-588a6e4c-3722-4f0a-8983-f3869f930378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:27.523817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 738/1477] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:27.546085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc44ef-7981-4f81-87aa-80c01a26f812 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:27.548172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5ddc44ef-7981-4f81-87aa-80c01a26f812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:27.670896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc44ef-7981-4f81-87aa-80c01a26f812 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:27.719029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:27.764196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f8e60-7abf-4c07-a295-d08804c70e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:27.764854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f8e60-7abf-4c07-a295-d08804c70e3d admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:27.765140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f8e60-7abf-4c07-a295-d08804c70e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:27.765412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f8e60-7abf-4c07-a295-d08804c70e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:27.766130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f8e60-7abf-4c07-a295-d08804c70e3d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:27.767959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-048f8e60-7abf-4c07-a295-d08804c70e3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:47:27.767959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jul 15 15:47:27.767959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: Jul 15 15:47:27.769111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f8e60-7abf-4c07-a295-d08804c70e3d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:27.787873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 740/1478] 10.222.0.98 () {56 vars in 1108 bytes} [Fri Jul 15 15:47:27 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:47:29.837849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66b662-0322-47ea-8e81-036b44e6938a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:29.838403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66b662-0322-47ea-8e81-036b44e6938a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:29.838403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66b662-0322-47ea-8e81-036b44e6938a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:29.839892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 739/1479] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:29.845231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2180c-03fe-4e9f-ae92-bc3a98c71235 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:29.845652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2180c-03fe-4e9f-ae92-bc3a98c71235 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:29.845792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2180c-03fe-4e9f-ae92-bc3a98c71235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:29.911965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-dde2180c-03fe-4e9f-ae92-bc3a98c71235 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:29.913128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dde2180c-03fe-4e9f-ae92-bc3a98c71235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:30.012825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 741/1480] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:30.056079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66b662-0322-47ea-8e81-036b44e6938a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:30.057683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ea66b662-0322-47ea-8e81-036b44e6938a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:30.068551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:30.203558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66b662-0322-47ea-8e81-036b44e6938a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:30.283929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:30.317708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356285a0-899e-4e91-86d3-5d136b675c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:30.318363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356285a0-899e-4e91-86d3-5d136b675c0a service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:30.318651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356285a0-899e-4e91-86d3-5d136b675c0a service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:30.318932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356285a0-899e-4e91-86d3-5d136b675c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:30.320613 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-356285a0-899e-4e91-86d3-5d136b675c0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:30.411573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356285a0-899e-4e91-86d3-5d136b675c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:30.413219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356285a0-899e-4e91-86d3-5d136b675c0a service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:30.431712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 740/1481] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:32.669310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd277b-145d-4079-a20f-75206a3d1cc3 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:32.669830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd277b-145d-4079-a20f-75206a3d1cc3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:32.669830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd277b-145d-4079-a20f-75206a3d1cc3 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:32.671195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 742/1482] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:32.676968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59cecc-a9c1-4407-870c-4258c60873af None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:32.677377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59cecc-a9c1-4407-870c-4258c60873af None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:32.677541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59cecc-a9c1-4407-870c-4258c60873af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:32.756248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5e59cecc-a9c1-4407-870c-4258c60873af None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:32.757497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5e59cecc-a9c1-4407-870c-4258c60873af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:32.804642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:32.856459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 741/1483] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:32.892784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd277b-145d-4079-a20f-75206a3d1cc3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:32.894317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d9bd277b-145d-4079-a20f-75206a3d1cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:32.917764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:32.998532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd277b-145d-4079-a20f-75206a3d1cc3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:33.074169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef708df-66e5-46b5-80cd-43bd13bbfe3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:33.074827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef708df-66e5-46b5-80cd-43bd13bbfe3e service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:33.075116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef708df-66e5-46b5-80cd-43bd13bbfe3e service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:33.075437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef708df-66e5-46b5-80cd-43bd13bbfe3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:33.077082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1ef708df-66e5-46b5-80cd-43bd13bbfe3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:33.151643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:33.189318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef708df-66e5-46b5-80cd-43bd13bbfe3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:33.190768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef708df-66e5-46b5-80cd-43bd13bbfe3e service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:33.211584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 743/1484] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:35.285910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a35f5-22f4-4c91-8bbc-9ed1548df3dd None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:35.286405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a35f5-22f4-4c91-8bbc-9ed1548df3dd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:35.286511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a35f5-22f4-4c91-8bbc-9ed1548df3dd None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:35.287897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 742/1485] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:35.293151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2c13-9ed4-44e5-8d6b-27c40f703db0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:35.293552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2c13-9ed4-44e5-8d6b-27c40f703db0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:35.293723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2c13-9ed4-44e5-8d6b-27c40f703db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:35.358394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-41ce2c13-9ed4-44e5-8d6b-27c40f703db0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:35.359634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-41ce2c13-9ed4-44e5-8d6b-27c40f703db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:35.469768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 744/1486] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:47:35.507471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a35f5-22f4-4c91-8bbc-9ed1548df3dd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:35.509081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c49a35f5-22f4-4c91-8bbc-9ed1548df3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:35.611870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a35f5-22f4-4c91-8bbc-9ed1548df3dd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:35.682088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:35.683864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89304d-1129-416d-96fb-ba17777ac373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:35.684513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89304d-1129-416d-96fb-ba17777ac373 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:35.684798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89304d-1129-416d-96fb-ba17777ac373 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:35.685076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89304d-1129-416d-96fb-ba17777ac373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:35.686706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5f89304d-1129-416d-96fb-ba17777ac373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:35.798138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89304d-1129-416d-96fb-ba17777ac373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:35.799543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89304d-1129-416d-96fb-ba17777ac373 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:35.818210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 743/1487] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:37.921988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32653-b5e8-44e0-ae86-6427e985687d None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:37.922558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32653-b5e8-44e0-ae86-6427e985687d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:37.922558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32653-b5e8-44e0-ae86-6427e985687d None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:37.924516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 745/1488] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:37.929785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f307c-e90d-4838-ba87-40a1d06d98ba None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:37.930188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f307c-e90d-4838-ba87-40a1d06d98ba None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:37.930352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f307c-e90d-4838-ba87-40a1d06d98ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:37.997017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-946f307c-e90d-4838-ba87-40a1d06d98ba None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:37.998224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-946f307c-e90d-4838-ba87-40a1d06d98ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:38.051966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:38.096061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 744/1489] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:38.137036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d32653-b5e8-44e0-ae86-6427e985687d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:38.138557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e6d32653-b5e8-44e0-ae86-6427e985687d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:38.174746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:38.241116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d32653-b5e8-44e0-ae86-6427e985687d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:38.315205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a707f-6323-4af4-8441-18242e317eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:38.328626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a707f-6323-4af4-8441-18242e317eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:38.328626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a707f-6323-4af4-8441-18242e317eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:38.328626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a707f-6323-4af4-8441-18242e317eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:38.328626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-005a707f-6323-4af4-8441-18242e317eb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:38.405931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:38.435653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a707f-6323-4af4-8441-18242e317eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:38.437051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a707f-6323-4af4-8441-18242e317eb7 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:38.456741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 746/1490] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:40.892597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1766e-0a5d-42c1-bb73-cac9eda5388d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:40.893163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1766e-0a5d-42c1-bb73-cac9eda5388d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:40.893163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1766e-0a5d-42c1-bb73-cac9eda5388d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:40.894564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 745/1491] 10.222.0.98 () {54 vars in 926 bytes} [Fri Jul 15 15:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:47:40.899954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf042d-4981-4d98-a3ea-175d66d0b06d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:40.900350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf042d-4981-4d98-a3ea-175d66d0b06d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:40.900516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf042d-4981-4d98-a3ea-175d66d0b06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:40.965117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2ebf042d-4981-4d98-a3ea-175d66d0b06d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:40.966322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2ebf042d-4981-4d98-a3ea-175d66d0b06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:41.180635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 747/1492] 10.222.0.98 () {58 vars in 1023 bytes} [Fri Jul 15 15:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:41.219788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1766e-0a5d-42c1-bb73-cac9eda5388d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:41.221936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cbf1766e-0a5d-42c1-bb73-cac9eda5388d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:41.322507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1766e-0a5d-42c1-bb73-cac9eda5388d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:41.399812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a7ea46-d6aa-4f01-9785-abd730686a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:41.400463 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7ea46-d6aa-4f01-9785-abd730686a6f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:41.400744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7ea46-d6aa-4f01-9785-abd730686a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:41.401019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a7ea46-d6aa-4f01-9785-abd730686a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:41.402651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-37a7ea46-d6aa-4f01-9785-abd730686a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:41.412987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:41.513156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7ea46-d6aa-4f01-9785-abd730686a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:41.514622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a7ea46-d6aa-4f01-9785-abd730686a6f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:41.543402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 746/1493] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:51.178676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=82 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:51.540849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=50 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:18.577374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd407f2e-5040-4b2d-9421-872845269cd5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:18.577857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd407f2e-5040-4b2d-9421-872845269cd5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:18.577961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd407f2e-5040-4b2d-9421-872845269cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:18.650519 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cd407f2e-5040-4b2d-9421-872845269cd5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:18.651759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd407f2e-5040-4b2d-9421-872845269cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:18.748698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 748/1494] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:18.756675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfbffa-93ab-413c-ac49-c1cb34b1deaf None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:18.757181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfbffa-93ab-413c-ac49-c1cb34b1deaf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:18.757288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfbffa-93ab-413c-ac49-c1cb34b1deaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:18.829822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2fcfbffa-93ab-413c-ac49-c1cb34b1deaf None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:18.831020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2fcfbffa-93ab-413c-ac49-c1cb34b1deaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:18.927904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 747/1495] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:18.932557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd407f2e-5040-4b2d-9421-872845269cd5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:18.933762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd407f2e-5040-4b2d-9421-872845269cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:18.945961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd407f2e-5040-4b2d-9421-872845269cd5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:19.016958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:19.050096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:19.050860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:19.051144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:19.051451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:19.052165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:49:19.052469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:19.053515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:49:19.053515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jul 15 15:49:19.053515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: Jul 15 15:49:19.054398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:19.065788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 749/1496] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:49:19.074880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfbffa-93ab-413c-ac49-c1cb34b1deaf None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:19.084731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfbffa-93ab-413c-ac49-c1cb34b1deaf None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:19.145976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:19.179403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:19.180063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:19.180348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:19.180626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:19.181306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:19.183125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:49:19.183125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jul 15 15:49:19.183125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: Jul 15 15:49:19.184378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:19.315149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 748/1497] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:49:19 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:49:19.320075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:19.321607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:19.360167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc877f-f9bd-44d2-aa19-a48811a93281 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:19.455639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:19.456354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:19.456633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:19.456905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:19.457619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:19.458622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:49:19.458622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jul 15 15:49:19.458622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: Jul 15 15:49:19.459873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:19.577651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 750/1498] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:49:19 2022] POST /identity/v3/users => generated 290 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:49:19.581779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:19.583404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:19.646511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92fce2-f1cb-4262-9342-2d6663b81120 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:19.731636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:19.742253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:19.747822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:19.747822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:19.747822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:19.747822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:19.747822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:49:19.747822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jul 15 15:49:19.747822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: Jul 15 15:49:19.748447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:19.758488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 749/1499] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:49:19.762690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:19.764261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:19.777164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62365ad6-0a7e-467b-9452-23544466a204 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:19.802546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:19.879093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8588ef0-5ce9-4f37-801c-3d813a270c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:19.879723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8588ef0-5ce9-4f37-801c-3d813a270c5c admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:19.880013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8588ef0-5ce9-4f37-801c-3d813a270c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:19.880293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8588ef0-5ce9-4f37-801c-3d813a270c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:19.881030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8588ef0-5ce9-4f37-801c-3d813a270c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:19.882136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-f8588ef0-5ce9-4f37-801c-3d813a270c5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:49:19.882136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jul 15 15:49:19.882136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: Jul 15 15:49:19.883110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8588ef0-5ce9-4f37-801c-3d813a270c5c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:19.896890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 751/1500] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:49:19.901827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:19.903424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:19.937403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18018ac-355a-451e-8982-40aa29b10ff1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:20.029964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ef466-83a1-465d-8086-5201ea09435b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:20.030855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ef466-83a1-465d-8086-5201ea09435b admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:20.031136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ef466-83a1-465d-8086-5201ea09435b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:20.031456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ef466-83a1-465d-8086-5201ea09435b admin admin] PATH_INFO: `/v3/projects/db4bdd94799b4e8faffabb6dafb3bfd2/users/b18d554ec2094fc1878204dc93688b4f/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:20.055907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ef466-83a1-465d-8086-5201ea09435b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db4bdd94799b4e8faffabb6dafb3bfd2, user_id=b18d554ec2094fc1878204dc93688b4f, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:20.056963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-5d7ef466-83a1-465d-8086-5201ea09435b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:49:20.056963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jul 15 15:49:20.056963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: Jul 15 15:49:20.058963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ef466-83a1-465d-8086-5201ea09435b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:20.226935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 750/1501] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:49:19 2022] PUT /identity/v3/projects/db4bdd94799b4e8faffabb6dafb3bfd2/users/b18d554ec2094fc1878204dc93688b4f/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:49:20.239808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968d7ac-acc0-408d-8c43-d2c5dec86a43 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:20.240132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968d7ac-acc0-408d-8c43-d2c5dec86a43 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:20.240445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968d7ac-acc0-408d-8c43-d2c5dec86a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:20.294442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8968d7ac-acc0-408d-8c43-d2c5dec86a43 None None] MFA Rules not processed for user `b18d554ec2094fc1878204dc93688b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:20.295687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8968d7ac-acc0-408d-8c43-d2c5dec86a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:20.371930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 752/1502] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:49:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:20.385707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc2667-22e3-42a4-ac66-575f58c9c97a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:20.386077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc2667-22e3-42a4-ac66-575f58c9c97a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:20.386375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc2667-22e3-42a4-ac66-575f58c9c97a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:20.387811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 751/1503] 10.222.0.98 () {54 vars in 963 bytes} [Fri Jul 15 15:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:49:20.392688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a832d30-042f-4a7a-893a-be7078caa314 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:20.392983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a832d30-042f-4a7a-893a-be7078caa314 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:20.393264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a832d30-042f-4a7a-893a-be7078caa314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:20.423482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:20.466552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5a832d30-042f-4a7a-893a-be7078caa314 None None] MFA Rules not processed for user `68015ac9febe42ef917ca58fae7b1c26`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:20.467795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5a832d30-042f-4a7a-893a-be7078caa314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:20.547735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 753/1504] 10.222.0.98 () {58 vars in 1060 bytes} [Fri Jul 15 15:49:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:20.555879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc2667-22e3-42a4-ac66-575f58c9c97a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:20.557388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ffcc2667-22e3-42a4-ac66-575f58c9c97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:20.607854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:20.779980 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc2667-22e3-42a4-ac66-575f58c9c97a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:20.852321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:20.852907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:20.853196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:20.853466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:20.855093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:20.953186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:20.954656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:20.958794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:20.974064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 752/1505] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:49:21.018220 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22376e93-2151-4997-b423-d4d0b720328a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.022596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22376e93-2151-4997-b423-d4d0b720328a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.022596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22376e93-2151-4997-b423-d4d0b720328a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.022596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 754/1506] 10.222.0.98 () {52 vars in 829 bytes} [Fri Jul 15 15:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:49:21.023842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:21.025062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:21.126057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94c24f-50ca-4980-869d-da6e5bd8aee2 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:21.216803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:21.217681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.217965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.218241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] PATH_INFO: `/v3/users/b18d554ec2094fc1878204dc93688b4f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.232178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18d554ec2094fc1878204dc93688b4f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:21.233243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:49:21.233243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jul 15 15:49:21.233243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: Jul 15 15:49:21.234467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:21.245691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:21.548530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18d554ec2094fc1878204dc93688b4f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:21.567444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18d554ec2094fc1878204dc93688b4f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:21.567998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18d554ec2094fc1878204dc93688b4f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:21.568421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18d554ec2094fc1878204dc93688b4f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:21.580505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18d554ec2094fc1878204dc93688b4f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:21.661438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 753/1507] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:49:21 2022] DELETE /identity/v3/users/b18d554ec2094fc1878204dc93688b4f => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:49:21.665888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee767b15-18b6-4a94-abb7-32fa97c8f4ba None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.666182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee767b15-18b6-4a94-abb7-32fa97c8f4ba None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.666460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee767b15-18b6-4a94-abb7-32fa97c8f4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.722352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ee767b15-18b6-4a94-abb7-32fa97c8f4ba None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:21.723642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ee767b15-18b6-4a94-abb7-32fa97c8f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:21.798281 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:21.811520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 755/1508] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:21.822226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:21.823492 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:21.922171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4fbfe-dca7-4774-be3e-a823456eb8a8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:21.976914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:21.995674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfabeb-cc80-43bf-b71a-bf7562203c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:21.996332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfabeb-cc80-43bf-b71a-bf7562203c2b service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.996617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfabeb-cc80-43bf-b71a-bf7562203c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.996894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfabeb-cc80-43bf-b71a-bf7562203c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.998522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-82cfabeb-cc80-43bf-b71a-bf7562203c2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:22.111138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfabeb-cc80-43bf-b71a-bf7562203c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:22.112634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfabeb-cc80-43bf-b71a-bf7562203c2b service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:22.132359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 754/1509] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:49:22.606095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee767b15-18b6-4a94-abb7-32fa97c8f4ba None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:22.607372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ee767b15-18b6-4a94-abb7-32fa97c8f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:22.619850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee767b15-18b6-4a94-abb7-32fa97c8f4ba None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:22.716230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:22.717094 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:22.717375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:22.717647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] PATH_INFO: `/v3/projects/db4bdd94799b4e8faffabb6dafb3bfd2` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:22.727948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db4bdd94799b4e8faffabb6dafb3bfd2)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:22.728977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:49:22.728977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jul 15 15:49:22.728977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: Jul 15 15:49:22.730177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:22.890248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db4bdd94799b4e8faffabb6dafb3bfd2', 'request_id': 'req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:22.896261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db4bdd94799b4e8faffabb6dafb3bfd2', 'request_id': 'req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:22.993635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db4bdd94799b4e8faffabb6dafb3bfd2', 'request_id': 'req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:23.023985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] The token cache is being invalidate because project db4bdd94799b4e8faffabb6dafb3bfd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:49:23.024275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:23.024801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-df897cec-5b2b-43f9-b4c1-9787a4b5dff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:23.026159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 756/1510] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:49:22 2022] DELETE /identity/v3/projects/db4bdd94799b4e8faffabb6dafb3bfd2 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:49:32.154188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=62 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:33.021934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:51:10.776279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c15d07-115b-474b-b3e5-c38f511ee3ec None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:10.776885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c15d07-115b-474b-b3e5-c38f511ee3ec None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:10.776885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c15d07-115b-474b-b3e5-c38f511ee3ec None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:10.778238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:10.783206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6760b-6b4d-48f7-bd3c-d9d50c5ab894 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:10.783674 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6760b-6b4d-48f7-bd3c-d9d50c5ab894 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:10.783840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6760b-6b4d-48f7-bd3c-d9d50c5ab894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:10.863739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-43b6760b-6b4d-48f7-bd3c-d9d50c5ab894 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:10.864988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-43b6760b-6b4d-48f7-bd3c-d9d50c5ab894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:10.968689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:11.121899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c15d07-115b-474b-b3e5-c38f511ee3ec None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:11.124021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-76c15d07-115b-474b-b3e5-c38f511ee3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:11.245313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c15d07-115b-474b-b3e5-c38f511ee3ec None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:11.250842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa9a0-3de5-432c-bd20-6d78310d5e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:11.251517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa9a0-3de5-432c-bd20-6d78310d5e5e service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:11.251811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa9a0-3de5-432c-bd20-6d78310d5e5e service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:11.252095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa9a0-3de5-432c-bd20-6d78310d5e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:11.260317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa9a0-3de5-432c-bd20-6d78310d5e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:11.261922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa9a0-3de5-432c-bd20-6d78310d5e5e service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:11.271354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 756/1513] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:12.103319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20403f72-b380-4a97-b46a-af0504e8c28e None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:12.103882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20403f72-b380-4a97-b46a-af0504e8c28e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:12.103882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20403f72-b380-4a97-b46a-af0504e8c28e None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:12.105317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:12.110256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85719f2-cfa8-4547-8ab8-d95210c81f50 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:12.110746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85719f2-cfa8-4547-8ab8-d95210c81f50 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:12.110882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85719f2-cfa8-4547-8ab8-d95210c81f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:12.129241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f85719f2-cfa8-4547-8ab8-d95210c81f50 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:12.130416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f85719f2-cfa8-4547-8ab8-d95210c81f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:12.138867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:51:12.295279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20403f72-b380-4a97-b46a-af0504e8c28e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:12.310480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20403f72-b380-4a97-b46a-af0504e8c28e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:12.316043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b988d-bd53-4a78-aa4d-afda3f24be27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:12.318186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b988d-bd53-4a78-aa4d-afda3f24be27 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:12.318186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b988d-bd53-4a78-aa4d-afda3f24be27 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:12.318186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b988d-bd53-4a78-aa4d-afda3f24be27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:12.325408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b988d-bd53-4a78-aa4d-afda3f24be27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:12.327255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b988d-bd53-4a78-aa4d-afda3f24be27 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:12.336469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ffebc-8d26-4f01-9e61-10642539c947 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:12.336853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ffebc-8d26-4f01-9e61-10642539c947 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:12.336958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 759/1516] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:12.341849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ffebc-8d26-4f01-9e61-10642539c947 None None] PATH_INFO: `/v3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:12.344247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 758/1517] 10.222.0.75 () {54 vars in 925 bytes} [Fri Jul 15 15:51:12 2022] GET /identity/v3 => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:51:12.350878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc9ba-e377-4b68-889f-5d809563ec70 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:12.351169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc9ba-e377-4b68-889f-5d809563ec70 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:12.351483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fc9ba-e377-4b68-889f-5d809563ec70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:12.422158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-913fc9ba-e377-4b68-889f-5d809563ec70 None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:12.423412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-913fc9ba-e377-4b68-889f-5d809563ec70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:12.441405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:51:12.496442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 760/1518] 10.222.0.75 () {58 vars in 1015 bytes} [Fri Jul 15 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:21.687204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf9fca-d032-48c7-a7f2-e8bb40f7fb2d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:21.687870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf9fca-d032-48c7-a7f2-e8bb40f7fb2d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:21.687870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf9fca-d032-48c7-a7f2-e8bb40f7fb2d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:21.689109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 759/1519] 10.222.0.98 () {54 vars in 960 bytes} [Fri Jul 15 15:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:21.693943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543154f5-1cb3-4c43-9c8a-1aa276f1f3b5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:21.694384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543154f5-1cb3-4c43-9c8a-1aa276f1f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:21.694534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543154f5-1cb3-4c43-9c8a-1aa276f1f3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:21.712950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-543154f5-1cb3-4c43-9c8a-1aa276f1f3b5 None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:21.714145 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-543154f5-1cb3-4c43-9c8a-1aa276f1f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:21.722699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 761/1520] 10.222.0.98 () {58 vars in 1057 bytes} [Fri Jul 15 15:51:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:21.730347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf9fca-d032-48c7-a7f2-e8bb40f7fb2d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:21.744376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf9fca-d032-48c7-a7f2-e8bb40f7fb2d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:21.749456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e5366-1ed0-4b82-9c63-d0f93117a199 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:21.750084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e5366-1ed0-4b82-9c63-d0f93117a199 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:21.750374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e5366-1ed0-4b82-9c63-d0f93117a199 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:21.750654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e5366-1ed0-4b82-9c63-d0f93117a199 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:21.759977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e5366-1ed0-4b82-9c63-d0f93117a199 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:21.765463 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e5366-1ed0-4b82-9c63-d0f93117a199 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:21.776495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 760/1521] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:23.590078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f892edb-dd4d-40c4-b86d-105b9f94a61e None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:23.590636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f892edb-dd4d-40c4-b86d-105b9f94a61e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:23.590636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f892edb-dd4d-40c4-b86d-105b9f94a61e None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:23.592082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:23.597128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237aae7e-570c-488b-8f17-49ec49a10520 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:23.597557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237aae7e-570c-488b-8f17-49ec49a10520 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:23.597723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237aae7e-570c-488b-8f17-49ec49a10520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:23.615389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-237aae7e-570c-488b-8f17-49ec49a10520 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:23.616588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-237aae7e-570c-488b-8f17-49ec49a10520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:23.624393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:23.789012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f892edb-dd4d-40c4-b86d-105b9f94a61e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:23.803233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f892edb-dd4d-40c4-b86d-105b9f94a61e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:23.811465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979027a7-7d60-463e-8027-beb4b8220cfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:23.811465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979027a7-7d60-463e-8027-beb4b8220cfa service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:23.811465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979027a7-7d60-463e-8027-beb4b8220cfa service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:23.811465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979027a7-7d60-463e-8027-beb4b8220cfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:23.818599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979027a7-7d60-463e-8027-beb4b8220cfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:23.820252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979027a7-7d60-463e-8027-beb4b8220cfa service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:23.830553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 763/1524] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:24.649576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cde6d-9a7d-469e-97f8-1d6f0e3c0362 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:24.650057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cde6d-9a7d-469e-97f8-1d6f0e3c0362 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:24.650160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cde6d-9a7d-469e-97f8-1d6f0e3c0362 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:24.651597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:24.656516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f83f5-79f9-4787-b2ec-fa96996536db None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:24.656932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f83f5-79f9-4787-b2ec-fa96996536db None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:24.657085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f83f5-79f9-4787-b2ec-fa96996536db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:24.674807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ee9f83f5-79f9-4787-b2ec-fa96996536db None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:24.675982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ee9f83f5-79f9-4787-b2ec-fa96996536db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:24.684439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:24.837627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cde6d-9a7d-469e-97f8-1d6f0e3c0362 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:24.851761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cde6d-9a7d-469e-97f8-1d6f0e3c0362 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:24.857582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14faa5b1-ce9d-416a-b3f3-8457a78b5a9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:24.858247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faa5b1-ce9d-416a-b3f3-8457a78b5a9f service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:24.858533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faa5b1-ce9d-416a-b3f3-8457a78b5a9f service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:24.858839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faa5b1-ce9d-416a-b3f3-8457a78b5a9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:24.870721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14faa5b1-ce9d-416a-b3f3-8457a78b5a9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:24.872417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14faa5b1-ce9d-416a-b3f3-8457a78b5a9f service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:24.882054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 763/1527] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:29.103622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4603993-ea43-4f9b-918b-5205e146f1fa None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:29.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4603993-ea43-4f9b-918b-5205e146f1fa None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:29.104210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4603993-ea43-4f9b-918b-5205e146f1fa None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:29.105555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:29.110672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f5f30-4f71-4e09-a8f8-45b17a1e806c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:29.111063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f5f30-4f71-4e09-a8f8-45b17a1e806c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:29.111234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f5f30-4f71-4e09-a8f8-45b17a1e806c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:29.129424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c44f5f30-4f71-4e09-a8f8-45b17a1e806c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:29.130635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c44f5f30-4f71-4e09-a8f8-45b17a1e806c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:29.139071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:29.175370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4603993-ea43-4f9b-918b-5205e146f1fa None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:29.177053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f4603993-ea43-4f9b-918b-5205e146f1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:29.254680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4603993-ea43-4f9b-918b-5205e146f1fa None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:29.258221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928569d-13f1-436e-bd80-fa7587481e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:29.259501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928569d-13f1-436e-bd80-fa7587481e05 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:29.259796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928569d-13f1-436e-bd80-fa7587481e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:29.260070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928569d-13f1-436e-bd80-fa7587481e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:29.268872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928569d-13f1-436e-bd80-fa7587481e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:29.270288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928569d-13f1-436e-bd80-fa7587481e05 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:29.279860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 766/1530] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:31.266352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988770f-da3d-496c-a280-7f29f12046f9 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:31.266916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988770f-da3d-496c-a280-7f29f12046f9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:31.266916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988770f-da3d-496c-a280-7f29f12046f9 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:31.268342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:31.273445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66308d-9a8e-4892-ba39-e86ce9be72e0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:31.273899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66308d-9a8e-4892-ba39-e86ce9be72e0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:31.274003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66308d-9a8e-4892-ba39-e86ce9be72e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:31.291665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6b66308d-9a8e-4892-ba39-e86ce9be72e0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:31.292895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6b66308d-9a8e-4892-ba39-e86ce9be72e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:31.300891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:31.337209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988770f-da3d-496c-a280-7f29f12046f9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:31.347856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988770f-da3d-496c-a280-7f29f12046f9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:31.351339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2ac27-9ca7-4476-b814-3ded2cc49dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:31.351976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2ac27-9ca7-4476-b814-3ded2cc49dfc service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:31.352292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2ac27-9ca7-4476-b814-3ded2cc49dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:31.352572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2ac27-9ca7-4476-b814-3ded2cc49dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:31.361289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2ac27-9ca7-4476-b814-3ded2cc49dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:31.362723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2ac27-9ca7-4476-b814-3ded2cc49dfc service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:31.372476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 766/1533] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:34.978012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bc7e3-1cc8-48c7-ba28-c5cf2f612ff2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:34.978589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bc7e3-1cc8-48c7-ba28-c5cf2f612ff2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:34.978589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bc7e3-1cc8-48c7-ba28-c5cf2f612ff2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:34.980131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:34.985162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a999b3a-9ba4-4242-ade6-443f89b84a38 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:34.985569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a999b3a-9ba4-4242-ade6-443f89b84a38 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:34.985726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a999b3a-9ba4-4242-ade6-443f89b84a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:34.991929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=81 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:51:35.004191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1a999b3a-9ba4-4242-ade6-443f89b84a38 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:35.005318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1a999b3a-9ba4-4242-ade6-443f89b84a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:35.013445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:35.320755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bc7e3-1cc8-48c7-ba28-c5cf2f612ff2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:35.322440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-014bc7e3-1cc8-48c7-ba28-c5cf2f612ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:35.389129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:51:35.402646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bc7e3-1cc8-48c7-ba28-c5cf2f612ff2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:35.406433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf211a1-dfa9-49db-9de6-4c1fcfdf1cab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:35.407078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf211a1-dfa9-49db-9de6-4c1fcfdf1cab service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:35.407738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf211a1-dfa9-49db-9de6-4c1fcfdf1cab service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:35.407738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf211a1-dfa9-49db-9de6-4c1fcfdf1cab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:35.416801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf211a1-dfa9-49db-9de6-4c1fcfdf1cab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:35.418219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf211a1-dfa9-49db-9de6-4c1fcfdf1cab service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:35.428134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 769/1536] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:37.470883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998bc4-11a4-4926-8318-b835d7e1fe89 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:37.471462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998bc4-11a4-4926-8318-b835d7e1fe89 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:37.471462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998bc4-11a4-4926-8318-b835d7e1fe89 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:37.472871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:51:37.478152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28f342-17ce-438b-81af-e8a4d5421e86 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:37.478580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28f342-17ce-438b-81af-e8a4d5421e86 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:37.478719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28f342-17ce-438b-81af-e8a4d5421e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:37.496917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5d28f342-17ce-438b-81af-e8a4d5421e86 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:51:37.498121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5d28f342-17ce-438b-81af-e8a4d5421e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:37.506150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:51:37.811679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82998bc4-11a4-4926-8318-b835d7e1fe89 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:51:37.814154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-82998bc4-11a4-4926-8318-b835d7e1fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:51:37.828893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82998bc4-11a4-4926-8318-b835d7e1fe89 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:51:37.832481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daade2-78dd-4f9d-b04f-74b7f44fe2ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:51:37.833133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daade2-78dd-4f9d-b04f-74b7f44fe2ff service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:51:37.833423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daade2-78dd-4f9d-b04f-74b7f44fe2ff service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:51:37.833705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daade2-78dd-4f9d-b04f-74b7f44fe2ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:51:37.842959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daade2-78dd-4f9d-b04f-74b7f44fe2ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:51:37.844441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daade2-78dd-4f9d-b04f-74b7f44fe2ff service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:51:37.854484 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 769/1539] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:51:47.491358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=11 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:51:47.849588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=8 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:11.085629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216b0d-624d-4d78-a736-ca8a983fe2f8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.086291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216b0d-624d-4d78-a736-ca8a983fe2f8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.086291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17216b0d-624d-4d78-a736-ca8a983fe2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.114908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-17216b0d-624d-4d78-a736-ca8a983fe2f8 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.116180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-17216b0d-624d-4d78-a736-ca8a983fe2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.128128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 771/1540] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:11.136870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c7a5-e526-4ebc-9241-54fa8289af3a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.137828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c7a5-e526-4ebc-9241-54fa8289af3a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.138358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c7a5-e526-4ebc-9241-54fa8289af3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.170150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-cb19c7a5-e526-4ebc-9241-54fa8289af3a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.171788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cb19c7a5-e526-4ebc-9241-54fa8289af3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.180260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 770/1541] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:11.185173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216b0d-624d-4d78-a736-ca8a983fe2f8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.202059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17216b0d-624d-4d78-a736-ca8a983fe2f8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.208466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.209341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.210370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.210868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.211788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:11.212267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.214047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.234207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 772/1542] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:11.241960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19c7a5-e526-4ebc-9241-54fa8289af3a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.258806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19c7a5-e526-4ebc-9241-54fa8289af3a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.267714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37975ae-a6da-48a6-b4a4-8f445da933ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.268404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37975ae-a6da-48a6-b4a4-8f445da933ff admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.268701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37975ae-a6da-48a6-b4a4-8f445da933ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.268980 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37975ae-a6da-48a6-b4a4-8f445da933ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.269690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37975ae-a6da-48a6-b4a4-8f445da933ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.271671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37975ae-a6da-48a6-b4a4-8f445da933ff admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.422092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 771/1543] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:11.427068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.446693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f98b8-949c-41ae-905b-61bd51f034e0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.525552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0b721-84fc-404a-b50d-6c3b886c8293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.526687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b721-84fc-404a-b50d-6c3b886c8293 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.527246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b721-84fc-404a-b50d-6c3b886c8293 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.527785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0b721-84fc-404a-b50d-6c3b886c8293 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.529018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0b721-84fc-404a-b50d-6c3b886c8293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.531308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0b721-84fc-404a-b50d-6c3b886c8293 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.540155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f404e-446e-44b3-8e33-b14c1e9c27fc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.540850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f404e-446e-44b3-8e33-b14c1e9c27fc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.541370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f404e-446e-44b3-8e33-b14c1e9c27fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.577692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d93f404e-446e-44b3-8e33-b14c1e9c27fc None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.579347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d93f404e-446e-44b3-8e33-b14c1e9c27fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.594646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 772/1544] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:11.603164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73c81-b345-4241-b345-cce73c91a87b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.603768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73c81-b345-4241-b345-cce73c91a87b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.604290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f73c81-b345-4241-b345-cce73c91a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.623323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 773/1545] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:11.623323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25582-f7d3-4f81-9417-0f3d6520795a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.623323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25582-f7d3-4f81-9417-0f3d6520795a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.623323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25582-f7d3-4f81-9417-0f3d6520795a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.633477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-08f73c81-b345-4241-b345-cce73c91a87b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.639436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-08f73c81-b345-4241-b345-cce73c91a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.650748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0fd25582-f7d3-4f81-9417-0f3d6520795a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.652134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0fd25582-f7d3-4f81-9417-0f3d6520795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.654119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 773/1546] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:53:11.655152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f73c81-b345-4241-b345-cce73c91a87b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.675414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f73c81-b345-4241-b345-cce73c91a87b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.675612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 774/1547] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:11.675612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd25582-f7d3-4f81-9417-0f3d6520795a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.684170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1a715-cdd3-4b6f-939a-de90b245f07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.685142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1a715-cdd3-4b6f-939a-de90b245f07f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.685811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1a715-cdd3-4b6f-939a-de90b245f07f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.686285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1a715-cdd3-4b6f-939a-de90b245f07f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.687228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1a715-cdd3-4b6f-939a-de90b245f07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.689019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd25582-f7d3-4f81-9417-0f3d6520795a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.689312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1a715-cdd3-4b6f-939a-de90b245f07f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.707337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.707337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.707337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.707337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.707800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:11.707800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.709376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.712264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 774/1548] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:11.719322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79fa8b-a5b0-4fd3-b75e-db93f870b230 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.719972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79fa8b-a5b0-4fd3-b75e-db93f870b230 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.720607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79fa8b-a5b0-4fd3-b75e-db93f870b230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.741083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 775/1549] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:11.742115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.751319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1a79fa8b-a5b0-4fd3-b75e-db93f870b230 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.752920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1a79fa8b-a5b0-4fd3-b75e-db93f870b230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.761444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 775/1550] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:11.763146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b28b9a-f6d0-48a5-8f47-b33a6f896eb9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.763685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b28b9a-f6d0-48a5-8f47-b33a6f896eb9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.764143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612a0b1-2b0d-48bb-9875-4eec6c5e38c8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.764399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b28b9a-f6d0-48a5-8f47-b33a6f896eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.797034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d2b28b9a-f6d0-48a5-8f47-b33a6f896eb9 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.797599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7735-18a6-4406-89be-013e7b81c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.798397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7735-18a6-4406-89be-013e7b81c02d admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.798764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7735-18a6-4406-89be-013e7b81c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.799409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7735-18a6-4406-89be-013e7b81c02d admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.799729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d2b28b9a-f6d0-48a5-8f47-b33a6f896eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.800280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b7735-18a6-4406-89be-013e7b81c02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.802763 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b7735-18a6-4406-89be-013e7b81c02d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.808547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 776/1551] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:11.809587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b28b9a-f6d0-48a5-8f47-b33a6f896eb9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.826801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 776/1552] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:11.827832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7735-18a6-4406-89be-013e7b81c02d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.832572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b28b9a-f6d0-48a5-8f47-b33a6f896eb9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.845670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7735-18a6-4406-89be-013e7b81c02d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.849570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41129f-f013-4932-a84c-1ab50c434860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.850902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41129f-f013-4932-a84c-1ab50c434860 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.851201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41129f-f013-4932-a84c-1ab50c434860 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.851883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41129f-f013-4932-a84c-1ab50c434860 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.852603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41129f-f013-4932-a84c-1ab50c434860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.856033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41129f-f013-4932-a84c-1ab50c434860 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.860811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c2d03-db04-4000-b18b-c900f7be07cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.861761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c2d03-db04-4000-b18b-c900f7be07cd admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.862274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c2d03-db04-4000-b18b-c900f7be07cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.862784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c2d03-db04-4000-b18b-c900f7be07cd admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.864025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c2d03-db04-4000-b18b-c900f7be07cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:11.865326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c2d03-db04-4000-b18b-c900f7be07cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.867331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c2d03-db04-4000-b18b-c900f7be07cd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.885577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 777/1553] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:11.889978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30830067-23df-4941-9daa-01ce5bc88ec7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.889978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30830067-23df-4941-9daa-01ce5bc88ec7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.889978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30830067-23df-4941-9daa-01ce5bc88ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.913197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-30830067-23df-4941-9daa-01ce5bc88ec7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.922269 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-30830067-23df-4941-9daa-01ce5bc88ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.923103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 778/1554] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:11.924360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222e513-509f-4b78-8c96-078ce20211d3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.924651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222e513-509f-4b78-8c96-078ce20211d3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.924933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222e513-509f-4b78-8c96-078ce20211d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.947247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d222e513-509f-4b78-8c96-078ce20211d3 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.947247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d222e513-509f-4b78-8c96-078ce20211d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:11.952165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 779/1555] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:11.952898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222e513-509f-4b78-8c96-078ce20211d3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:11.962385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222e513-509f-4b78-8c96-078ce20211d3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:11.967575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb2ea3-8060-45ea-bf97-855311692ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:11.968444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb2ea3-8060-45ea-bf97-855311692ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.968756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb2ea3-8060-45ea-bf97-855311692ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.969007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb2ea3-8060-45ea-bf97-855311692ebe admin admin] PATH_INFO: `/v3/projects/1111d08f11fa40189407b3d0ce6df136/users/28c12b17379f4a4baf5d8b24f1aecbc8/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.977690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 777/1556] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/projects => generated 391 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:11.977690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c2b58-47e3-48c7-8370-3468625db35a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:11.977690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c2b58-47e3-48c7-8370-3468625db35a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:11.977690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c2b58-47e3-48c7-8370-3468625db35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:11.984300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb2ea3-8060-45ea-bf97-855311692ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1111d08f11fa40189407b3d0ce6df136, user_id=28c12b17379f4a4baf5d8b24f1aecbc8, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:11.985339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-49eb2ea3-8060-45ea-bf97-855311692ebe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:53:11.985339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jul 15 15:53:11.985339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: Jul 15 15:53:11.987276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb2ea3-8060-45ea-bf97-855311692ebe admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:11.995818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b58c2b58-47e3-48c7-8370-3468625db35a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:11.997016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b58c2b58-47e3-48c7-8370-3468625db35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:12.057978 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 778/1557] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:12.059087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c2b58-47e3-48c7-8370-3468625db35a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.059256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 780/1558] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:11 2022] PUT /identity/v3/projects/1111d08f11fa40189407b3d0ce6df136/users/28c12b17379f4a4baf5d8b24f1aecbc8/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:12.060587 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8150a7-1908-48aa-b6d2-2eca9f9e8831 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.060881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8150a7-1908-48aa-b6d2-2eca9f9e8831 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.061161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8150a7-1908-48aa-b6d2-2eca9f9e8831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.078368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c2b58-47e3-48c7-8370-3468625db35a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.087803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2d8150a7-1908-48aa-b6d2-2eca9f9e8831 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:12.090955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2d8150a7-1908-48aa-b6d2-2eca9f9e8831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:12.139472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:12.149152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510cfe-5321-429a-adb6-ca9edd1c16b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.149825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510cfe-5321-429a-adb6-ca9edd1c16b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.150108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510cfe-5321-429a-adb6-ca9edd1c16b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.150385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510cfe-5321-429a-adb6-ca9edd1c16b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.151063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510cfe-5321-429a-adb6-ca9edd1c16b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.153806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510cfe-5321-429a-adb6-ca9edd1c16b0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.155127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 781/1559] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:12.156454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaf120-b69c-442a-8a7d-cd452a3461b2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.156751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaf120-b69c-442a-8a7d-cd452a3461b2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.157083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeaf120-b69c-442a-8a7d-cd452a3461b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.170064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:12.176429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3aeaf120-b69c-442a-8a7d-cd452a3461b2 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:12.177972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3aeaf120-b69c-442a-8a7d-cd452a3461b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:12.186736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 782/1560] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:12.189687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeaf120-b69c-442a-8a7d-cd452a3461b2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.202909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeaf120-b69c-442a-8a7d-cd452a3461b2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.205606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.206271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.206555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.206837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.207587 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:12.207900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.209549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.220899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 779/1561] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:12.229103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 783/1562] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:12.229294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a391ae-a033-4d0a-b73d-79b28d764112 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.229294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a391ae-a033-4d0a-b73d-79b28d764112 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.229504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.229606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a391ae-a033-4d0a-b73d-79b28d764112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.233048 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d3ca9-e8a1-45c8-ac23-14e78ef3efad admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.242820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-50a391ae-a033-4d0a-b73d-79b28d764112 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:12.244106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-50a391ae-a033-4d0a-b73d-79b28d764112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:12.289789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba593a-6f8d-4bde-85ca-edb2033241be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.290462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba593a-6f8d-4bde-85ca-edb2033241be admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.290745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba593a-6f8d-4bde-85ca-edb2033241be admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.291025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba593a-6f8d-4bde-85ca-edb2033241be admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.291739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba593a-6f8d-4bde-85ca-edb2033241be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.293671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba593a-6f8d-4bde-85ca-edb2033241be admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.304851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 780/1563] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:12.306147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5d70-889a-415b-ba6d-090877c32ade None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.306444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5d70-889a-415b-ba6d-090877c32ade None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.306732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5d70-889a-415b-ba6d-090877c32ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.332138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-79da5d70-889a-415b-ba6d-090877c32ade None None] MFA Rules not processed for user `28c12b17379f4a4baf5d8b24f1aecbc8`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:12.333351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-79da5d70-889a-415b-ba6d-090877c32ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:12.398935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 784/1564] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:12.399177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 781/1565] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:12.399308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0478b9-9f61-4576-bf6f-1ea734d89a53 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.399308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0478b9-9f61-4576-bf6f-1ea734d89a53 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.399308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0478b9-9f61-4576-bf6f-1ea734d89a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.399638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abd5f-2ccd-40ae-bcf1-30a4a18f8619 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.399638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abd5f-2ccd-40ae-bcf1-30a4a18f8619 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.399824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abd5f-2ccd-40ae-bcf1-30a4a18f8619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.419187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-520abd5f-2ccd-40ae-bcf1-30a4a18f8619 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:12.420421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-520abd5f-2ccd-40ae-bcf1-30a4a18f8619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:12.422667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6b0478b9-9f61-4576-bf6f-1ea734d89a53 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:12.423921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6b0478b9-9f61-4576-bf6f-1ea734d89a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:12.428501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 782/1566] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:12.429824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1b08a-bcc2-424b-8221-491f8bc5da60 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.430117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1b08a-bcc2-424b-8221-491f8bc5da60 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.430400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1b08a-bcc2-424b-8221-491f8bc5da60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.434425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 785/1567] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:12.435198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0478b9-9f61-4576-bf6f-1ea734d89a53 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.452830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0478b9-9f61-4576-bf6f-1ea734d89a53 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.453175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-94f1b08a-bcc2-424b-8221-491f8bc5da60 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:12.454218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-94f1b08a-bcc2-424b-8221-491f8bc5da60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:12.458228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd0faf-b958-439c-885e-5a07855d29a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.459074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd0faf-b958-439c-885e-5a07855d29a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.459509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd0faf-b958-439c-885e-5a07855d29a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.460134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd0faf-b958-439c-885e-5a07855d29a4 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.460968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd0faf-b958-439c-885e-5a07855d29a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.462561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 783/1568] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:12.463410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1b08a-bcc2-424b-8221-491f8bc5da60 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.463848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd0faf-b958-439c-885e-5a07855d29a4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.473301 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1b08a-bcc2-424b-8221-491f8bc5da60 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.479320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c51461-2a3c-4c2b-861c-210ef4bc0a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.479986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c51461-2a3c-4c2b-861c-210ef4bc0a1e admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.480810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c51461-2a3c-4c2b-861c-210ef4bc0a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.481095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c51461-2a3c-4c2b-861c-210ef4bc0a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.481785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c51461-2a3c-4c2b-861c-210ef4bc0a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.485730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c51461-2a3c-4c2b-861c-210ef4bc0a1e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.533436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 786/1569] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:12.534274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd0faf-b958-439c-885e-5a07855d29a4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.544138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd0faf-b958-439c-885e-5a07855d29a4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.548702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.549334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.549617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.549898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.550608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:12.550917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.552552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.564221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 784/1570] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:12.564221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c51461-2a3c-4c2b-861c-210ef4bc0a1e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.574533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 787/1571] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:12.574533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.574707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1c2c8-b266-4708-b604-55ec7c93c541 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.612060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:12.628899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c51461-2a3c-4c2b-861c-210ef4bc0a1e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.631817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28527e6-1f4c-4bbd-939a-aae674e85233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.635354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28527e6-1f4c-4bbd-939a-aae674e85233 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.635354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28527e6-1f4c-4bbd-939a-aae674e85233 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.635354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28527e6-1f4c-4bbd-939a-aae674e85233 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.635655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6072e6-1b24-455b-a15b-31a1b4c64b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.635767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28527e6-1f4c-4bbd-939a-aae674e85233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.635858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6072e6-1b24-455b-a15b-31a1b4c64b74 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.635858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6072e6-1b24-455b-a15b-31a1b4c64b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.635858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6072e6-1b24-455b-a15b-31a1b4c64b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.636192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28527e6-1f4c-4bbd-939a-aae674e85233 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.636288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6072e6-1b24-455b-a15b-31a1b4c64b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.637616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6072e6-1b24-455b-a15b-31a1b4c64b74 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.645464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 788/1572] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:12.646329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28527e6-1f4c-4bbd-939a-aae674e85233 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.658586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28527e6-1f4c-4bbd-939a-aae674e85233 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.668540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.668540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.668540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.668540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.668958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:12.668958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.668958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.679992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 789/1573] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:12.680944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.686942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:12.691036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e633b-2aee-4272-8971-b11b55c3bd48 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.695774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.696416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.696705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.696982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.697699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:12.698005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.699621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.711932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 790/1574] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:12.712741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.722816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc8501-8fc5-4409-8772-138fda638ff7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.727484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.728140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.728417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.728705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.733109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 785/1575] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:12.733333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:12.733333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.733665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6072e6-1b24-455b-a15b-31a1b4c64b74 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.733835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.749848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 791/1576] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:12.750668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.762282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e665c-ef69-43f4-95f4-6173effe0813 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.800843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6072e6-1b24-455b-a15b-31a1b4c64b74 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.805727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbec257-c902-48b3-8568-bc750f2d629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.806429 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbec257-c902-48b3-8568-bc750f2d629f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.806721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbec257-c902-48b3-8568-bc750f2d629f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.807003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbec257-c902-48b3-8568-bc750f2d629f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.807756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbec257-c902-48b3-8568-bc750f2d629f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.809565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbec257-c902-48b3-8568-bc750f2d629f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.821181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 786/1577] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:12.822008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbec257-c902-48b3-8568-bc750f2d629f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.830115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5e26d-9a4e-402b-a0b7-8ce962246ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.830799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5e26d-9a4e-402b-a0b7-8ce962246ddf admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.831085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5e26d-9a4e-402b-a0b7-8ce962246ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.831393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5e26d-9a4e-402b-a0b7-8ce962246ddf admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.832094 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5e26d-9a4e-402b-a0b7-8ce962246ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.832734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbec257-c902-48b3-8568-bc750f2d629f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.834054 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5e26d-9a4e-402b-a0b7-8ce962246ddf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.837707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84d950-2027-4e90-aafd-4ae6cd6a6ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.838357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84d950-2027-4e90-aafd-4ae6cd6a6ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.838645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84d950-2027-4e90-aafd-4ae6cd6a6ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.838926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84d950-2027-4e90-aafd-4ae6cd6a6ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.840137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84d950-2027-4e90-aafd-4ae6cd6a6ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.842057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84d950-2027-4e90-aafd-4ae6cd6a6ba3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.944621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 792/1578] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:12.945644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5e26d-9a4e-402b-a0b7-8ce962246ddf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.956036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5e26d-9a4e-402b-a0b7-8ce962246ddf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:12.960930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2be332-6a82-4c2e-8b2a-721bde1f65dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:12.961565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2be332-6a82-4c2e-8b2a-721bde1f65dd admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:12.961852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2be332-6a82-4c2e-8b2a-721bde1f65dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:12.962179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2be332-6a82-4c2e-8b2a-721bde1f65dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:12.962835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2be332-6a82-4c2e-8b2a-721bde1f65dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:12.964462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2be332-6a82-4c2e-8b2a-721bde1f65dd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:12.976105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 787/1579] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:12.981679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 793/1580] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:12.981803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84d950-2027-4e90-aafd-4ae6cd6a6ba3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.981895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2be332-6a82-4c2e-8b2a-721bde1f65dd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:12.987546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2be332-6a82-4c2e-8b2a-721bde1f65dd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.041380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84d950-2027-4e90-aafd-4ae6cd6a6ba3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.044039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.044747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.045108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.045423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.046111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.046222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de391ae-c5ff-4e4e-854a-ab1fc7bb54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.046861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de391ae-c5ff-4e4e-854a-ab1fc7bb54a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.047133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:53:13.047133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jul 15 15:53:13.047133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: Jul 15 15:53:13.047411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de391ae-c5ff-4e4e-854a-ab1fc7bb54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.047499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de391ae-c5ff-4e4e-854a-ab1fc7bb54a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.048108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de391ae-c5ff-4e4e-854a-ab1fc7bb54a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.048362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.050003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de391ae-c5ff-4e4e-854a-ab1fc7bb54a5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.169583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 794/1581] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:13.170477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.186968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 788/1582] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:12 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:13.194237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de391ae-c5ff-4e4e-854a-ab1fc7bb54a5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.222804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:13.229469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:13.246863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d7a39-9a2e-41bc-8858-a5756850d0ee admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.251567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc46727-0fc8-4b5d-ac04-18218cd64fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.252236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc46727-0fc8-4b5d-ac04-18218cd64fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.252527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc46727-0fc8-4b5d-ac04-18218cd64fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.252808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc46727-0fc8-4b5d-ac04-18218cd64fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.253494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc46727-0fc8-4b5d-ac04-18218cd64fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.253761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de391ae-c5ff-4e4e-854a-ab1fc7bb54a5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.255464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc46727-0fc8-4b5d-ac04-18218cd64fd4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.259028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40e6b5-6cbd-42da-8408-16ff26e10814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.263243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40e6b5-6cbd-42da-8408-16ff26e10814 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.263243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40e6b5-6cbd-42da-8408-16ff26e10814 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.263243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40e6b5-6cbd-42da-8408-16ff26e10814 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.263243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40e6b5-6cbd-42da-8408-16ff26e10814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.263624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40e6b5-6cbd-42da-8408-16ff26e10814 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.368300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 795/1583] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:13 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:13.369061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc46727-0fc8-4b5d-ac04-18218cd64fd4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.379670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc46727-0fc8-4b5d-ac04-18218cd64fd4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.385343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 789/1584] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:13 2022] POST /identity/v3/projects => generated 371 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:13.385343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40e6b5-6cbd-42da-8408-16ff26e10814 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.441871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a8da-e617-49ae-8b50-ceb6e0b3269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.442560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a8da-e617-49ae-8b50-ceb6e0b3269e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.442846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a8da-e617-49ae-8b50-ceb6e0b3269e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.443126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a8da-e617-49ae-8b50-ceb6e0b3269e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.443883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38a8da-e617-49ae-8b50-ceb6e0b3269e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.445492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38a8da-e617-49ae-8b50-ceb6e0b3269e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.454718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40e6b5-6cbd-42da-8408-16ff26e10814 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.456276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 796/1585] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:13.457044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a8da-e617-49ae-8b50-ceb6e0b3269e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.460921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b2176-a1d8-4b3e-aa73-977440aa67e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.461589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2176-a1d8-4b3e-aa73-977440aa67e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.461873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2176-a1d8-4b3e-aa73-977440aa67e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.462149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b2176-a1d8-4b3e-aa73-977440aa67e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.462907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b2176-a1d8-4b3e-aa73-977440aa67e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.464875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b2176-a1d8-4b3e-aa73-977440aa67e8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.467750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a8da-e617-49ae-8b50-ceb6e0b3269e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.472395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c763fe-591e-44e9-ac11-65243e50cb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.473109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c763fe-591e-44e9-ac11-65243e50cb10 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.473347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c763fe-591e-44e9-ac11-65243e50cb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.473628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c763fe-591e-44e9-ac11-65243e50cb10 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.474314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c763fe-591e-44e9-ac11-65243e50cb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.476260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c763fe-591e-44e9-ac11-65243e50cb10 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.481667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 790/1586] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:13.482749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b2176-a1d8-4b3e-aa73-977440aa67e8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.492622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b2176-a1d8-4b3e-aa73-977440aa67e8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.497869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d231b-e203-4b30-b3fb-ec5a49d1cccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.498835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d231b-e203-4b30-b3fb-ec5a49d1cccf admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.499127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d231b-e203-4b30-b3fb-ec5a49d1cccf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.499443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d231b-e203-4b30-b3fb-ec5a49d1cccf admin admin] PATH_INFO: `/v3/projects/44cff0ae69e84de486ac2a6866ea6acb/users/9005cb23317a4a05b58e99284824135e/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.516053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d231b-e203-4b30-b3fb-ec5a49d1cccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cff0ae69e84de486ac2a6866ea6acb, user_id=9005cb23317a4a05b58e99284824135e, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.518766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d231b-e203-4b30-b3fb-ec5a49d1cccf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.562348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 797/1587] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:13 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:13.563151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c763fe-591e-44e9-ac11-65243e50cb10 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.572430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c763fe-591e-44e9-ac11-65243e50cb10 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.577625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63466e-1bac-4f47-8b04-054513b43b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.578258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63466e-1bac-4f47-8b04-054513b43b73 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.578545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63466e-1bac-4f47-8b04-054513b43b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.578826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63466e-1bac-4f47-8b04-054513b43b73 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.579523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63466e-1bac-4f47-8b04-054513b43b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.581435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63466e-1bac-4f47-8b04-054513b43b73 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.602263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 791/1588] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:13 2022] PUT /identity/v3/projects/44cff0ae69e84de486ac2a6866ea6acb/users/9005cb23317a4a05b58e99284824135e/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:13.603182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d231b-e203-4b30-b3fb-ec5a49d1cccf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.663571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d231b-e203-4b30-b3fb-ec5a49d1cccf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.668784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.669453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.669744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.670027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.670713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.671773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:53:13.671773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jul 15 15:53:13.671773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: Jul 15 15:53:13.673005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.689229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 798/1589] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:13 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:13.690137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63466e-1bac-4f47-8b04-054513b43b73 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.699732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63466e-1bac-4f47-8b04-054513b43b73 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.704404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacde3a-42b3-4efd-83d1-f2b910eb7919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.705026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacde3a-42b3-4efd-83d1-f2b910eb7919 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.705316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacde3a-42b3-4efd-83d1-f2b910eb7919 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.705594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacde3a-42b3-4efd-83d1-f2b910eb7919 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.706352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacde3a-42b3-4efd-83d1-f2b910eb7919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.707991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacde3a-42b3-4efd-83d1-f2b910eb7919 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.718367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 799/1590] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:13.719155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacde3a-42b3-4efd-83d1-f2b910eb7919 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.728947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacde3a-42b3-4efd-83d1-f2b910eb7919 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.734354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d523-aa5b-4960-a413-f59de4f2f6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.735021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d523-aa5b-4960-a413-f59de4f2f6bd admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.735324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d523-aa5b-4960-a413-f59de4f2f6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.735609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d523-aa5b-4960-a413-f59de4f2f6bd admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.736297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d523-aa5b-4960-a413-f59de4f2f6bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.738246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d523-aa5b-4960-a413-f59de4f2f6bd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.784618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 792/1591] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:13 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:13.785541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.797734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad60fc5-89fa-44b3-bd0e-b19662b78127 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.802307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e18cc-2965-4dab-82e2-80257ce2e22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.803746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e18cc-2965-4dab-82e2-80257ce2e22b admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.803746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e18cc-2965-4dab-82e2-80257ce2e22b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.803746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e18cc-2965-4dab-82e2-80257ce2e22b admin admin] PATH_INFO: `/v3/projects/c02b7fdd2ebe442cad31bc5d84fb7ba5/users/d03681f0b895409e847b3a40bf43ac45/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.816807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:13.819778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e18cc-2965-4dab-82e2-80257ce2e22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02b7fdd2ebe442cad31bc5d84fb7ba5, user_id=d03681f0b895409e847b3a40bf43ac45, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.822490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e18cc-2965-4dab-82e2-80257ce2e22b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.858439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 800/1592] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:13 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:13.859271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d523-aa5b-4960-a413-f59de4f2f6bd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.868877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d523-aa5b-4960-a413-f59de4f2f6bd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.874183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc966a41-c309-4ab2-b90c-d0de683c6792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.874821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc966a41-c309-4ab2-b90c-d0de683c6792 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.875108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc966a41-c309-4ab2-b90c-d0de683c6792 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.875421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc966a41-c309-4ab2-b90c-d0de683c6792 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.876126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc966a41-c309-4ab2-b90c-d0de683c6792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.877713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc966a41-c309-4ab2-b90c-d0de683c6792 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.888546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 801/1593] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:13.889375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc966a41-c309-4ab2-b90c-d0de683c6792 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.898798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc966a41-c309-4ab2-b90c-d0de683c6792 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.903437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224b1e1-a28c-480f-b5bd-04d97b46feff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.904067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224b1e1-a28c-480f-b5bd-04d97b46feff admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.904350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224b1e1-a28c-480f-b5bd-04d97b46feff admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.904633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224b1e1-a28c-480f-b5bd-04d97b46feff admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.905316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224b1e1-a28c-480f-b5bd-04d97b46feff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.906875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224b1e1-a28c-480f-b5bd-04d97b46feff admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.916789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 802/1594] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:13.918336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5105c-6d47-4654-98ba-eefc94ef98fa None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.918644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5105c-6d47-4654-98ba-eefc94ef98fa None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.918906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5105c-6d47-4654-98ba-eefc94ef98fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.925637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 793/1595] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:13 2022] PUT /identity/v3/projects/c02b7fdd2ebe442cad31bc5d84fb7ba5/users/d03681f0b895409e847b3a40bf43ac45/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:13.926552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e18cc-2965-4dab-82e2-80257ce2e22b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:13.938920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8aa5105c-6d47-4654-98ba-eefc94ef98fa None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:13.940446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8aa5105c-6d47-4654-98ba-eefc94ef98fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:13.977838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:13.988952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e18cc-2965-4dab-82e2-80257ce2e22b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:13.993680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3825e23-4fee-4633-ac22-8db8284124ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:13.994411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3825e23-4fee-4633-ac22-8db8284124ee admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:13.994698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3825e23-4fee-4633-ac22-8db8284124ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:13.994979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3825e23-4fee-4633-ac22-8db8284124ee admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:13.995737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3825e23-4fee-4633-ac22-8db8284124ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:13.998074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3825e23-4fee-4633-ac22-8db8284124ee admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:13.999592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 803/1596] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:14.000282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5105c-6d47-4654-98ba-eefc94ef98fa None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.009791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 794/1597] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:14.010047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5105c-6d47-4654-98ba-eefc94ef98fa None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.010826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3825e23-4fee-4633-ac22-8db8284124ee admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.016355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00e9c-ae5d-430d-bc70-82bf80b48622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.017702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00e9c-ae5d-430d-bc70-82bf80b48622 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.017702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00e9c-ae5d-430d-bc70-82bf80b48622 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.017924 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c00e9c-ae5d-430d-bc70-82bf80b48622 admin admin] PATH_INFO: `/v3/projects/28bafc988eda41adbb5956605cd81472/users/6e1caab2e6be4fac96de16bc499bb727/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.023118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3825e23-4fee-4633-ac22-8db8284124ee admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.028272 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc569c1a-371b-4c69-bde3-fc6b385c37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.028959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc569c1a-371b-4c69-bde3-fc6b385c37ef admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.029246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc569c1a-371b-4c69-bde3-fc6b385c37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.029523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc569c1a-371b-4c69-bde3-fc6b385c37ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.030263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc569c1a-371b-4c69-bde3-fc6b385c37ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.031914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc569c1a-371b-4c69-bde3-fc6b385c37ef admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.033579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c00e9c-ae5d-430d-bc70-82bf80b48622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bafc988eda41adbb5956605cd81472, user_id=6e1caab2e6be4fac96de16bc499bb727, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.036351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c00e9c-ae5d-430d-bc70-82bf80b48622 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.042250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 795/1598] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:14.043909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc569c1a-371b-4c69-bde3-fc6b385c37ef admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.054430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc569c1a-371b-4c69-bde3-fc6b385c37ef admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.058973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037b663-a518-4e50-90c5-4047870f59a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.059631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b663-a518-4e50-90c5-4047870f59a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.059918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b663-a518-4e50-90c5-4047870f59a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.060194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037b663-a518-4e50-90c5-4047870f59a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.060895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037b663-a518-4e50-90c5-4047870f59a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.062488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037b663-a518-4e50-90c5-4047870f59a3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.073044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 796/1599] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:14.073807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037b663-a518-4e50-90c5-4047870f59a3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.082894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037b663-a518-4e50-90c5-4047870f59a3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.087718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d572fa-2dac-4c91-8849-1133178a739f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.092274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d572fa-2dac-4c91-8849-1133178a739f admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.092274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d572fa-2dac-4c91-8849-1133178a739f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.092274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d572fa-2dac-4c91-8849-1133178a739f admin admin] PATH_INFO: `/v3/projects/a5644a8a1ae540e2b260c3d1bb2bea0f/users/8e90cd544b0745809f118964cbdc66bb/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.104026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d572fa-2dac-4c91-8849-1133178a739f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5644a8a1ae540e2b260c3d1bb2bea0f, user_id=8e90cd544b0745809f118964cbdc66bb, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.107427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d572fa-2dac-4c91-8849-1133178a739f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.117332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 804/1600] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:13 2022] PUT /identity/v3/projects/28bafc988eda41adbb5956605cd81472/users/6e1caab2e6be4fac96de16bc499bb727/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:14.118279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00e9c-ae5d-430d-bc70-82bf80b48622 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.173727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 797/1601] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:14 2022] PUT /identity/v3/projects/a5644a8a1ae540e2b260c3d1bb2bea0f/users/8e90cd544b0745809f118964cbdc66bb/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:14.175486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd710c6-72f6-4858-a3a4-d630efd257e0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.175807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd710c6-72f6-4858-a3a4-d630efd257e0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.176090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd710c6-72f6-4858-a3a4-d630efd257e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.182598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c00e9c-ae5d-430d-bc70-82bf80b48622 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.210742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0fd710c6-72f6-4858-a3a4-d630efd257e0 None None] MFA Rules not processed for user `d03681f0b895409e847b3a40bf43ac45`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:14.212238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0fd710c6-72f6-4858-a3a4-d630efd257e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:14.238951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e439f-fdda-41a5-87c5-e195d8251d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.239716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e439f-fdda-41a5-87c5-e195d8251d90 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.239950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e439f-fdda-41a5-87c5-e195d8251d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.240230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e439f-fdda-41a5-87c5-e195d8251d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.240954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e439f-fdda-41a5-87c5-e195d8251d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.242558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e439f-fdda-41a5-87c5-e195d8251d90 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.252300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 805/1602] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:14.256274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e439f-fdda-41a5-87c5-e195d8251d90 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.276400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 798/1603] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:14.277077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd710c6-72f6-4858-a3a4-d630efd257e0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.287647 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd710c6-72f6-4858-a3a4-d630efd257e0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.292269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005904-af21-4866-9978-2d022f811c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.293629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005904-af21-4866-9978-2d022f811c9e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.293948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005904-af21-4866-9978-2d022f811c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.294231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1005904-af21-4866-9978-2d022f811c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.294950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1005904-af21-4866-9978-2d022f811c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.296687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1005904-af21-4866-9978-2d022f811c9e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.306617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 799/1604] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:14.307438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005904-af21-4866-9978-2d022f811c9e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.317193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e439f-fdda-41a5-87c5-e195d8251d90 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.319157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1005904-af21-4866-9978-2d022f811c9e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.320847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded3eba-2f74-45fe-a0fe-670b0df02316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.321502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded3eba-2f74-45fe-a0fe-670b0df02316 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.321790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded3eba-2f74-45fe-a0fe-670b0df02316 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.322073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded3eba-2f74-45fe-a0fe-670b0df02316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.323810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53a4ed-e4c5-4548-96bd-5dd7dda710e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.324471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53a4ed-e4c5-4548-96bd-5dd7dda710e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.324772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53a4ed-e4c5-4548-96bd-5dd7dda710e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.325049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53a4ed-e4c5-4548-96bd-5dd7dda710e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.325753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53a4ed-e4c5-4548-96bd-5dd7dda710e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.327506 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53a4ed-e4c5-4548-96bd-5dd7dda710e5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.331277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded3eba-2f74-45fe-a0fe-670b0df02316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.332794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded3eba-2f74-45fe-a0fe-670b0df02316 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.337553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 800/1605] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:14.338341 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53a4ed-e4c5-4548-96bd-5dd7dda710e5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.343595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 806/1606] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:14.344572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded3eba-2f74-45fe-a0fe-670b0df02316 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.348692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53a4ed-e4c5-4548-96bd-5dd7dda710e5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.350872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:14.353462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a4ae1-d97b-47cf-b93c-a0ee4c2afe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.354127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4ae1-d97b-47cf-b93c-a0ee4c2afe32 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.354423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4ae1-d97b-47cf-b93c-a0ee4c2afe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.354703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4ae1-d97b-47cf-b93c-a0ee4c2afe32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.354821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded3eba-2f74-45fe-a0fe-670b0df02316 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.355644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a4ae1-d97b-47cf-b93c-a0ee4c2afe32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.357266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a4ae1-d97b-47cf-b93c-a0ee4c2afe32 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.359716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ced058-e5ca-4d2a-881c-3002547b38c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.360379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ced058-e5ca-4d2a-881c-3002547b38c4 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.364068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:14.364174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ced058-e5ca-4d2a-881c-3002547b38c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.364174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ced058-e5ca-4d2a-881c-3002547b38c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.364174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ced058-e5ca-4d2a-881c-3002547b38c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.364174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ced058-e5ca-4d2a-881c-3002547b38c4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.371752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 801/1607] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:14.374225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 807/1608] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:14.375811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ced058-e5ca-4d2a-881c-3002547b38c4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.376533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b429-76a9-46fa-89d2-ce4b8f7e19f1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.376830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b429-76a9-46fa-89d2-ce4b8f7e19f1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.377110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b429-76a9-46fa-89d2-ce4b8f7e19f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.389162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ced058-e5ca-4d2a-881c-3002547b38c4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.393807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24338aeb-1830-4155-a053-3a5b993a5f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.394531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3cf0b429-76a9-46fa-89d2-ce4b8f7e19f1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:14.394860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24338aeb-1830-4155-a053-3a5b993a5f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.395164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24338aeb-1830-4155-a053-3a5b993a5f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.395507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24338aeb-1830-4155-a053-3a5b993a5f55 admin admin] PATH_INFO: `/v3/projects/c5b5c3ebb2ec42988f6203ec34392dc6/users/2ff316f85f3e40178a0e7354a123e483/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.395938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3cf0b429-76a9-46fa-89d2-ce4b8f7e19f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:14.403960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 802/1609] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:14.404571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0b429-76a9-46fa-89d2-ce4b8f7e19f1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.410903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24338aeb-1830-4155-a053-3a5b993a5f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b5c3ebb2ec42988f6203ec34392dc6, user_id=2ff316f85f3e40178a0e7354a123e483, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.413702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24338aeb-1830-4155-a053-3a5b993a5f55 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.415843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0b429-76a9-46fa-89d2-ce4b8f7e19f1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.421309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362567-e3f5-4cbb-9304-7c7c8ee304d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.421996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362567-e3f5-4cbb-9304-7c7c8ee304d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.422267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362567-e3f5-4cbb-9304-7c7c8ee304d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.422549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362567-e3f5-4cbb-9304-7c7c8ee304d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.423241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4362567-e3f5-4cbb-9304-7c7c8ee304d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.425179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4362567-e3f5-4cbb-9304-7c7c8ee304d3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.510890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 808/1610] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:14 2022] PUT /identity/v3/projects/c5b5c3ebb2ec42988f6203ec34392dc6/users/2ff316f85f3e40178a0e7354a123e483/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:14.525141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24338aeb-1830-4155-a053-3a5b993a5f55 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.540034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 803/1611] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:14 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:14.540933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362567-e3f5-4cbb-9304-7c7c8ee304d3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.607998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24338aeb-1830-4155-a053-3a5b993a5f55 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.611906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362567-e3f5-4cbb-9304-7c7c8ee304d3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.613319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c763-bb5d-4964-8261-1311bfbcb86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.614204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092f9d0-6b8f-41a0-8554-e2012d2d6799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.614374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c763-bb5d-4964-8261-1311bfbcb86c admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.614567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c763-bb5d-4964-8261-1311bfbcb86c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.614849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66c763-bb5d-4964-8261-1311bfbcb86c admin admin] PATH_INFO: `/v3/projects/7df1ad577ddf4dc8a9a75ad1a1337ffe/users/ac8dfa5c97594e74a6839d91b292adcd/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.615483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092f9d0-6b8f-41a0-8554-e2012d2d6799 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.615842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092f9d0-6b8f-41a0-8554-e2012d2d6799 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.616148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092f9d0-6b8f-41a0-8554-e2012d2d6799 admin admin] PATH_INFO: `/v3/projects/93e4d5ff424c4e4fa5dd81c20cc920d0/users/78db558bb3ea4161b4a54e62f9ec3bd6/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.632251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66c763-bb5d-4964-8261-1311bfbcb86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df1ad577ddf4dc8a9a75ad1a1337ffe, user_id=ac8dfa5c97594e74a6839d91b292adcd, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.635277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092f9d0-6b8f-41a0-8554-e2012d2d6799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e4d5ff424c4e4fa5dd81c20cc920d0, user_id=78db558bb3ea4161b4a54e62f9ec3bd6, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.635516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66c763-bb5d-4964-8261-1311bfbcb86c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.639202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092f9d0-6b8f-41a0-8554-e2012d2d6799 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.716778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 809/1612] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:14 2022] PUT /identity/v3/projects/7df1ad577ddf4dc8a9a75ad1a1337ffe/users/ac8dfa5c97594e74a6839d91b292adcd/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:14.717843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c763-bb5d-4964-8261-1311bfbcb86c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.761302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 804/1613] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:14 2022] PUT /identity/v3/projects/93e4d5ff424c4e4fa5dd81c20cc920d0/users/78db558bb3ea4161b4a54e62f9ec3bd6/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:14.762543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092f9d0-6b8f-41a0-8554-e2012d2d6799 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.786020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66c763-bb5d-4964-8261-1311bfbcb86c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.831271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092f9d0-6b8f-41a0-8554-e2012d2d6799 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.836455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae77cd3-2ee8-4364-937a-4eb30a6ea94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.837358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77cd3-2ee8-4364-937a-4eb30a6ea94f admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.837656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77cd3-2ee8-4364-937a-4eb30a6ea94f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.837939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae77cd3-2ee8-4364-937a-4eb30a6ea94f admin admin] PATH_INFO: `/v3/projects/28bafc988eda41adbb5956605cd81472/users/6e1caab2e6be4fac96de16bc499bb727/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.840564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae77cd3-2ee8-4364-937a-4eb30a6ea94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bafc988eda41adbb5956605cd81472, user_id=6e1caab2e6be4fac96de16bc499bb727, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.843502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae77cd3-2ee8-4364-937a-4eb30a6ea94f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.856706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 805/1614] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:14 2022] PUT /identity/v3/projects/28bafc988eda41adbb5956605cd81472/users/6e1caab2e6be4fac96de16bc499bb727/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:14.857557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae77cd3-2ee8-4364-937a-4eb30a6ea94f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:14.860323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33954d89-a39a-458c-8ea1-9ec5ca2d727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.863356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33954d89-a39a-458c-8ea1-9ec5ca2d727b admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.863356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33954d89-a39a-458c-8ea1-9ec5ca2d727b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.863356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33954d89-a39a-458c-8ea1-9ec5ca2d727b admin admin] PATH_INFO: `/v3/projects/5677770f291d42e493b4ef0dd346d049/users/a93503cfdcad4ec2a40a4865a31270c0/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.866064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:14.887415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33954d89-a39a-458c-8ea1-9ec5ca2d727b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5677770f291d42e493b4ef0dd346d049, user_id=a93503cfdcad4ec2a40a4865a31270c0, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.893523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33954d89-a39a-458c-8ea1-9ec5ca2d727b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:14.918992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:14.925372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae77cd3-2ee8-4364-937a-4eb30a6ea94f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:14.929557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f8485-9052-43c4-a383-013b44ce2a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:14.930332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f8485-9052-43c4-a383-013b44ce2a0e service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.930621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f8485-9052-43c4-a383-013b44ce2a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.930994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f8485-9052-43c4-a383-013b44ce2a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.963590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 810/1615] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:14 2022] PUT /identity/v3/projects/5677770f291d42e493b4ef0dd346d049/users/a93503cfdcad4ec2a40a4865a31270c0/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:14.966575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f64025-9e06-4cf5-825a-4bc1df4bb7e5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:14.967125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f64025-9e06-4cf5-825a-4bc1df4bb7e5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:14.967600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f64025-9e06-4cf5-825a-4bc1df4bb7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:14.997365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-09f64025-9e06-4cf5-825a-4bc1df4bb7e5 None None] MFA Rules not processed for user `2ff316f85f3e40178a0e7354a123e483`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:14.998686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f8485-9052-43c4-a383-013b44ce2a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:14.998906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-09f64025-9e06-4cf5-825a-4bc1df4bb7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.000234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f8485-9052-43c4-a383-013b44ce2a0e service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.010876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 806/1616] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:15.015504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f8485-9052-43c4-a383-013b44ce2a0e service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.021902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f8485-9052-43c4-a383-013b44ce2a0e service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.079098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 811/1617] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:15.079098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104f241-9afb-43a0-b80a-9e50a0c84bda None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.079098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104f241-9afb-43a0-b80a-9e50a0c84bda None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.079098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104f241-9afb-43a0-b80a-9e50a0c84bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.097150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44880b88-d1d9-40c2-b454-85f3aa37bef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.097937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44880b88-d1d9-40c2-b454-85f3aa37bef3 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.098299 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44880b88-d1d9-40c2-b454-85f3aa37bef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.099160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44880b88-d1d9-40c2-b454-85f3aa37bef3 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.099891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44880b88-d1d9-40c2-b454-85f3aa37bef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.102152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44880b88-d1d9-40c2-b454-85f3aa37bef3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.106305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1104f241-9afb-43a0-b80a-9e50a0c84bda None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:15.107602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1104f241-9afb-43a0-b80a-9e50a0c84bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.119176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 812/1618] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:15.120673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0bb79-1fbe-4563-9a0b-9624cc9a9b2a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.120993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0bb79-1fbe-4563-9a0b-9624cc9a9b2a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.121277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0bb79-1fbe-4563-9a0b-9624cc9a9b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.145961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-94c0bb79-1fbe-4563-9a0b-9624cc9a9b2a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:15.147194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-94c0bb79-1fbe-4563-9a0b-9624cc9a9b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.156031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 813/1619] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:15.157350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e64c6-ed29-4303-8491-5334dae4caa7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.157649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e64c6-ed29-4303-8491-5334dae4caa7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.157930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e64c6-ed29-4303-8491-5334dae4caa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.184690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-788e64c6-ed29-4303-8491-5334dae4caa7 None None] MFA Rules not processed for user `6e1caab2e6be4fac96de16bc499bb727`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:15.185981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-788e64c6-ed29-4303-8491-5334dae4caa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.209474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 807/1620] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:15 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:15.211419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ce55b-22bf-4530-9546-217853937ad8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.211969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ce55b-22bf-4530-9546-217853937ad8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.212187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ce55b-22bf-4530-9546-217853937ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.250608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1e6ce55b-22bf-4530-9546-217853937ad8 None None] MFA Rules not processed for user `a93503cfdcad4ec2a40a4865a31270c0`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:15.252451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1e6ce55b-22bf-4530-9546-217853937ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.265715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 814/1621] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:15.271638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c0a3f-3ea8-4aa4-a362-82a41957b213 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.271932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c0a3f-3ea8-4aa4-a362-82a41957b213 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.272211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c0a3f-3ea8-4aa4-a362-82a41957b213 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.273984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 815/1622] 10.222.0.98 () {54 vars in 961 bytes} [Fri Jul 15 15:53:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:53:15.275025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c0a3f-3ea8-4aa4-a362-82a41957b213 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.276732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c00c0a3f-3ea8-4aa4-a362-82a41957b213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.332849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 808/1623] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:15.333850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ce55b-22bf-4530-9546-217853937ad8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.339279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1e6ce55b-22bf-4530-9546-217853937ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.339687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:15.351313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c0a3f-3ea8-4aa4-a362-82a41957b213 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.356696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a74e00-ae3d-491b-b8fb-1d88a7f082a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.357054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ce55b-22bf-4530-9546-217853937ad8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.357827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74e00-ae3d-491b-b8fb-1d88a7f082a1 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.358270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74e00-ae3d-491b-b8fb-1d88a7f082a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.358884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74e00-ae3d-491b-b8fb-1d88a7f082a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.361412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daebf49-001a-4fdf-876f-1899b6105b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.362296 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daebf49-001a-4fdf-876f-1899b6105b60 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.362644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daebf49-001a-4fdf-876f-1899b6105b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.362979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daebf49-001a-4fdf-876f-1899b6105b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.371548 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a74e00-ae3d-491b-b8fb-1d88a7f082a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.373721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a74e00-ae3d-491b-b8fb-1d88a7f082a1 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.374865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daebf49-001a-4fdf-876f-1899b6105b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.376403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daebf49-001a-4fdf-876f-1899b6105b60 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.388175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 816/1624] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:15.389041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a74e00-ae3d-491b-b8fb-1d88a7f082a1 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.398926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a74e00-ae3d-491b-b8fb-1d88a7f082a1 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.400080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 809/1625] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:15.401447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9872440-eddb-46c3-be73-f4a8152bf6b1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.401742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9872440-eddb-46c3-be73-f4a8152bf6b1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.403106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9872440-eddb-46c3-be73-f4a8152bf6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.408892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5124c-b684-45d8-b807-652d3dddc3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.413579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5124c-b684-45d8-b807-652d3dddc3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.413579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5124c-b684-45d8-b807-652d3dddc3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.413579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5124c-b684-45d8-b807-652d3dddc3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.413579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5124c-b684-45d8-b807-652d3dddc3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.413919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:15.414000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5124c-b684-45d8-b807-652d3dddc3a9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.428018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 817/1626] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:15.428018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5124c-b684-45d8-b807-652d3dddc3a9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.452228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c9872440-eddb-46c3-be73-f4a8152bf6b1 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:15.453610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c9872440-eddb-46c3-be73-f4a8152bf6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.503910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5124c-b684-45d8-b807-652d3dddc3a9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.508219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ba737-4828-42cc-934e-1b08f38b687f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.508883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba737-4828-42cc-934e-1b08f38b687f service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.511121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba737-4828-42cc-934e-1b08f38b687f service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.511121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba737-4828-42cc-934e-1b08f38b687f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.520361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ba737-4828-42cc-934e-1b08f38b687f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.521847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ba737-4828-42cc-934e-1b08f38b687f service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.533369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 818/1627] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:15.534782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1250e8-efbb-4d93-aad1-e9ad7f22f30f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.535089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1250e8-efbb-4d93-aad1-e9ad7f22f30f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.535198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 810/1628] 10.222.0.98 () {58 vars in 1058 bytes} [Fri Jul 15 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:15.535426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1250e8-efbb-4d93-aad1-e9ad7f22f30f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.539794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9872440-eddb-46c3-be73-f4a8152bf6b1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.540027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 819/1629] 10.222.0.98 () {54 vars in 961 bytes} [Fri Jul 15 15:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:53:15.547411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9872440-eddb-46c3-be73-f4a8152bf6b1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.549330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2804-cec8-4b96-8396-29b6ef088903 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.549983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2804-cec8-4b96-8396-29b6ef088903 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.550516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2804-cec8-4b96-8396-29b6ef088903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.552410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd7b81-ff4e-4a9c-93d1-e2768c468e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.553819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd7b81-ff4e-4a9c-93d1-e2768c468e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.554424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd7b81-ff4e-4a9c-93d1-e2768c468e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.554817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd7b81-ff4e-4a9c-93d1-e2768c468e93 admin admin] PATH_INFO: `/v3/projects/7af5872b9b0c47a1a0acc541624d5a06/users/0e7fc7233b2a4f928a4151aa2799f163/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.575663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd7b81-ff4e-4a9c-93d1-e2768c468e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af5872b9b0c47a1a0acc541624d5a06, user_id=0e7fc7233b2a4f928a4151aa2799f163, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.578262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b53b2804-cec8-4b96-8396-29b6ef088903 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:15.578798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd7b81-ff4e-4a9c-93d1-e2768c468e93 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.579512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b53b2804-cec8-4b96-8396-29b6ef088903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:15.601014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 820/1630] 10.222.0.98 () {58 vars in 1058 bytes} [Fri Jul 15 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:15.601014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b2804-cec8-4b96-8396-29b6ef088903 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.602208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b2804-cec8-4b96-8396-29b6ef088903 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.608807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd32e2-b790-4bda-8707-3843fc2bd8ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.609466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd32e2-b790-4bda-8707-3843fc2bd8ca service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.609751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd32e2-b790-4bda-8707-3843fc2bd8ca service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.610027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd32e2-b790-4bda-8707-3843fc2bd8ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.619999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd32e2-b790-4bda-8707-3843fc2bd8ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.622184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd32e2-b790-4bda-8707-3843fc2bd8ca service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.634406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 821/1631] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:15.635708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd32e2-b790-4bda-8707-3843fc2bd8ca service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.656511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd32e2-b790-4bda-8707-3843fc2bd8ca service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.670119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871599d-703c-4ec0-9b18-119efbbd897e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.671110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871599d-703c-4ec0-9b18-119efbbd897e service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.671530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871599d-703c-4ec0-9b18-119efbbd897e service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.671917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871599d-703c-4ec0-9b18-119efbbd897e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.691126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871599d-703c-4ec0-9b18-119efbbd897e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.694964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871599d-703c-4ec0-9b18-119efbbd897e service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.709493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 822/1632] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:15.734065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 811/1633] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:15 2022] PUT /identity/v3/projects/7af5872b9b0c47a1a0acc541624d5a06/users/0e7fc7233b2a4f928a4151aa2799f163/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:15.738004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871599d-703c-4ec0-9b18-119efbbd897e service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.754171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871599d-703c-4ec0-9b18-119efbbd897e service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.839610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bdf15-e30f-44ed-ac57-7a02b356ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.840533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bdf15-e30f-44ed-ac57-7a02b356ba7b admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.840958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bdf15-e30f-44ed-ac57-7a02b356ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.841382 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bdf15-e30f-44ed-ac57-7a02b356ba7b admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.842260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bdf15-e30f-44ed-ac57-7a02b356ba7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.854931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bdf15-e30f-44ed-ac57-7a02b356ba7b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.881930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd7b81-ff4e-4a9c-93d1-e2768c468e93 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.883652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 823/1634] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:15.884652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bdf15-e30f-44ed-ac57-7a02b356ba7b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:15.906856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bdf15-e30f-44ed-ac57-7a02b356ba7b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:15.917747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50226c-f15c-45e8-99f8-f1103212777a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:15.921625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50226c-f15c-45e8-99f8-f1103212777a admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.922062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50226c-f15c-45e8-99f8-f1103212777a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.922451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50226c-f15c-45e8-99f8-f1103212777a admin admin] PATH_INFO: `/v3/projects/7af5872b9b0c47a1a0acc541624d5a06/users/0e7fc7233b2a4f928a4151aa2799f163/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.927667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50226c-f15c-45e8-99f8-f1103212777a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af5872b9b0c47a1a0acc541624d5a06, user_id=0e7fc7233b2a4f928a4151aa2799f163, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:15.934657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50226c-f15c-45e8-99f8-f1103212777a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:15.958589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 824/1635] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:15 2022] PUT /identity/v3/projects/7af5872b9b0c47a1a0acc541624d5a06/users/0e7fc7233b2a4f928a4151aa2799f163/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:15.962766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff04720-3577-487e-8697-fc73129e3b15 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.981934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff04720-3577-487e-8697-fc73129e3b15 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.981934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff04720-3577-487e-8697-fc73129e3b15 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:15.981934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 825/1636] 10.222.0.98 () {54 vars in 912 bytes} [Fri Jul 15 15:53:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:53:15.981934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b60301-9dff-4c64-8f54-042bcc1f8215 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:15.981934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b60301-9dff-4c64-8f54-042bcc1f8215 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:15.981934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b60301-9dff-4c64-8f54-042bcc1f8215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:16.005602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-34b60301-9dff-4c64-8f54-042bcc1f8215 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:16.006699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-34b60301-9dff-4c64-8f54-042bcc1f8215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:16.010764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd7b81-ff4e-4a9c-93d1-e2768c468e93 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:16.082961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 826/1637] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:53:16.084405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ca1d-0e5a-4035-85ad-ebb954ed940c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:16.085125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ca1d-0e5a-4035-85ad-ebb954ed940c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:16.085282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ca1d-0e5a-4035-85ad-ebb954ed940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:16.115343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd481f-6273-4690-8dd0-f50344e51865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:16.116068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd481f-6273-4690-8dd0-f50344e51865 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:16.116364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd481f-6273-4690-8dd0-f50344e51865 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:16.116641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd481f-6273-4690-8dd0-f50344e51865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:16.139543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ea52ca1d-0e5a-4035-85ad-ebb954ed940c None None] MFA Rules not processed for user `0e7fc7233b2a4f928a4151aa2799f163`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:16.143756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ea52ca1d-0e5a-4035-85ad-ebb954ed940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:16.223277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd481f-6273-4690-8dd0-f50344e51865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:16.227211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd481f-6273-4690-8dd0-f50344e51865 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:16.267920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 812/1638] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:16.270604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd481f-6273-4690-8dd0-f50344e51865 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:16.282721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 827/1639] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:16.288631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71d921-aec3-42e4-8905-44c34f752c81 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:16.289715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd481f-6273-4690-8dd0-f50344e51865 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:16.293634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71d921-aec3-42e4-8905-44c34f752c81 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:16.293948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71d921-aec3-42e4-8905-44c34f752c81 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:16.295613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 828/1640] 10.222.0.98 () {54 vars in 912 bytes} [Fri Jul 15 15:53:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:53:16.301353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a1ba7-e250-435c-9a57-4ffc7cefe28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:16.301867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61506cea-4c02-49dc-8567-da28b7cdfbd0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:16.302164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61506cea-4c02-49dc-8567-da28b7cdfbd0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:16.302379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1ba7-e250-435c-9a57-4ffc7cefe28c service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:16.302497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61506cea-4c02-49dc-8567-da28b7cdfbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:16.302860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1ba7-e250-435c-9a57-4ffc7cefe28c service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:16.305852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1ba7-e250-435c-9a57-4ffc7cefe28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:16.335509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a1ba7-e250-435c-9a57-4ffc7cefe28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:16.338191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-61506cea-4c02-49dc-8567-da28b7cdfbd0 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:16.341175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-61506cea-4c02-49dc-8567-da28b7cdfbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:16.341478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a1ba7-e250-435c-9a57-4ffc7cefe28c service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:16.366277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 829/1641] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:53:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:53:16.371173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 813/1642] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:16.664917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:16.844649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:16.950561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61506cea-4c02-49dc-8567-da28b7cdfbd0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:16.977108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61506cea-4c02-49dc-8567-da28b7cdfbd0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:16.987374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aac8a2-3432-4d69-8cae-d1950a9b1572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:17.253396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aac8a2-3432-4d69-8cae-d1950a9b1572 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:17.253396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aac8a2-3432-4d69-8cae-d1950a9b1572 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:17.253396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aac8a2-3432-4d69-8cae-d1950a9b1572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:17.253396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aac8a2-3432-4d69-8cae-d1950a9b1572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:17.253396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aac8a2-3432-4d69-8cae-d1950a9b1572 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:17.253396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 830/1643] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:17.255527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a1ba7-e250-435c-9a57-4ffc7cefe28c service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:17.255527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a1ba7-e250-435c-9a57-4ffc7cefe28c service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:17.255527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a2408-6f13-4785-ac1e-1a11428c77b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:17.255527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a2408-6f13-4785-ac1e-1a11428c77b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:17.256412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a2408-6f13-4785-ac1e-1a11428c77b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:17.256412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a2408-6f13-4785-ac1e-1a11428c77b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:17.256412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a2408-6f13-4785-ac1e-1a11428c77b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:17.256412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a2408-6f13-4785-ac1e-1a11428c77b0 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:17.269733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 814/1644] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:18.236011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aac8a2-3432-4d69-8cae-d1950a9b1572 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:18.248428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aac8a2-3432-4d69-8cae-d1950a9b1572 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:18.257823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3cf54-bdcd-4a24-9033-c527b21af0f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:18.259029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3cf54-bdcd-4a24-9033-c527b21af0f4 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:18.259029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3cf54-bdcd-4a24-9033-c527b21af0f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:18.259327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3cf54-bdcd-4a24-9033-c527b21af0f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:18.375780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3cf54-bdcd-4a24-9033-c527b21af0f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:18.377374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3cf54-bdcd-4a24-9033-c527b21af0f4 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:18.420592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 831/1645] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:24.332427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee923058-3bc9-4ad1-90f4-d845202ba513 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:24.332927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee923058-3bc9-4ad1-90f4-d845202ba513 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:24.333005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee923058-3bc9-4ad1-90f4-d845202ba513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:24.394108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ee923058-3bc9-4ad1-90f4-d845202ba513 None None] MFA Rules not processed for user `9005cb23317a4a05b58e99284824135e`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:24.395421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ee923058-3bc9-4ad1-90f4-d845202ba513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:24.633814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 815/1646] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:24.646509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f0def-12a8-4350-92b1-f9cb7e4cb4f2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:24.646940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f0def-12a8-4350-92b1-f9cb7e4cb4f2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:24.647006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f0def-12a8-4350-92b1-f9cb7e4cb4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:24.740298 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3a6f0def-12a8-4350-92b1-f9cb7e4cb4f2 None None] MFA Rules not processed for user `9005cb23317a4a05b58e99284824135e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:24.741564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3a6f0def-12a8-4350-92b1-f9cb7e4cb4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:24.777478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 832/1647] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:24.817714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee923058-3bc9-4ad1-90f4-d845202ba513 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:24.837363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee923058-3bc9-4ad1-90f4-d845202ba513 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:24.845604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f778cd-93ab-4a5c-b42c-911acbdfea4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:24.845928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f778cd-93ab-4a5c-b42c-911acbdfea4a service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:24.846238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f778cd-93ab-4a5c-b42c-911acbdfea4a service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:24.846517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f778cd-93ab-4a5c-b42c-911acbdfea4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:24.870176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f778cd-93ab-4a5c-b42c-911acbdfea4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:24.872699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f778cd-93ab-4a5c-b42c-911acbdfea4a service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:24.887724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 816/1648] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:26.527240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2665-a129-47bc-81a0-5d42e4ee4b05 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:26.527802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2665-a129-47bc-81a0-5d42e4ee4b05 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:26.530535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2665-a129-47bc-81a0-5d42e4ee4b05 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:26.538811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 833/1649] 10.222.0.98 () {54 vars in 912 bytes} [Fri Jul 15 15:53:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:53:26.545217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494ac9e-3518-4d09-b6da-cb19f6c57ea4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:26.545659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494ac9e-3518-4d09-b6da-cb19f6c57ea4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:26.546008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494ac9e-3518-4d09-b6da-cb19f6c57ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:26.645740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d494ac9e-3518-4d09-b6da-cb19f6c57ea4 None None] MFA Rules not processed for user `9d20de9c921942d1afeb63e87272f782`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:26.647272 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d494ac9e-3518-4d09-b6da-cb19f6c57ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:26.729488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 817/1650] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:26.763428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c2665-a129-47bc-81a0-5d42e4ee4b05 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:26.783843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c2665-a129-47bc-81a0-5d42e4ee4b05 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:26.798984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ef01b-5e97-43fb-82af-ba8bfcedee9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:26.803411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef01b-5e97-43fb-82af-ba8bfcedee9a service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:26.804184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef01b-5e97-43fb-82af-ba8bfcedee9a service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:26.807546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ef01b-5e97-43fb-82af-ba8bfcedee9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:26.834898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ef01b-5e97-43fb-82af-ba8bfcedee9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:26.836800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ef01b-5e97-43fb-82af-ba8bfcedee9a service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:26.869073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 834/1651] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:27.426214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dd1d9-8527-4ba0-b0d8-2937ff7657cb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:27.426760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dd1d9-8527-4ba0-b0d8-2937ff7657cb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:27.426860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dd1d9-8527-4ba0-b0d8-2937ff7657cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:27.487220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:27.492142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ea7dd1d9-8527-4ba0-b0d8-2937ff7657cb None None] MFA Rules not processed for user `8e90cd544b0745809f118964cbdc66bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:27.496964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ea7dd1d9-8527-4ba0-b0d8-2937ff7657cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:27.654407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 818/1652] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:27.667984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31e6a-84ef-4b37-85d0-d3921755d3b1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:27.668453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31e6a-84ef-4b37-85d0-d3921755d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:27.668586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c31e6a-84ef-4b37-85d0-d3921755d3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:27.694323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:27.743833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-89c31e6a-84ef-4b37-85d0-d3921755d3b1 None None] MFA Rules not processed for user `8e90cd544b0745809f118964cbdc66bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:27.748530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-89c31e6a-84ef-4b37-85d0-d3921755d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:27.782656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 835/1653] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:27.792229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dd1d9-8527-4ba0-b0d8-2937ff7657cb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:27.834251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dd1d9-8527-4ba0-b0d8-2937ff7657cb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:27.885649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e701588-f7d2-44b3-b96c-c8027c2c65ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:27.888082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e701588-f7d2-44b3-b96c-c8027c2c65ba service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:27.888382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e701588-f7d2-44b3-b96c-c8027c2c65ba service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:27.888670 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e701588-f7d2-44b3-b96c-c8027c2c65ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:27.904394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e701588-f7d2-44b3-b96c-c8027c2c65ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:27.913414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e701588-f7d2-44b3-b96c-c8027c2c65ba service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:27.966978 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 819/1654] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:28.658731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70878aa-6eed-4936-b71a-90fb4a042a7c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:28.659022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70878aa-6eed-4936-b71a-90fb4a042a7c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:28.659756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70878aa-6eed-4936-b71a-90fb4a042a7c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:28.661762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 836/1655] 10.222.0.98 () {54 vars in 960 bytes} [Fri Jul 15 15:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:53:28.671317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2c22-1171-4260-85da-9ec01a6f6289 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:28.671764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2c22-1171-4260-85da-9ec01a6f6289 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:28.671835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2c22-1171-4260-85da-9ec01a6f6289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:28.718595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9ead2c22-1171-4260-85da-9ec01a6f6289 None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:28.720012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9ead2c22-1171-4260-85da-9ec01a6f6289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:28.953505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a06be9-9b1d-4227-b12b-ab6bf72670d2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:28.955085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a06be9-9b1d-4227-b12b-ab6bf72670d2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:28.955877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a06be9-9b1d-4227-b12b-ab6bf72670d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:28.964255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 820/1656] 10.222.0.98 () {58 vars in 1057 bytes} [Fri Jul 15 15:53:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:28.970573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead2c22-1171-4260-85da-9ec01a6f6289 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:28.990040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead2c22-1171-4260-85da-9ec01a6f6289 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:29.011687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533220c3-e4a9-4b71-84e5-ee9cf66556c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:29.012666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533220c3-e4a9-4b71-84e5-ee9cf66556c2 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:29.012964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533220c3-e4a9-4b71-84e5-ee9cf66556c2 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:29.013250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533220c3-e4a9-4b71-84e5-ee9cf66556c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:29.024483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-533220c3-e4a9-4b71-84e5-ee9cf66556c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:29.036783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-17a06be9-9b1d-4227-b12b-ab6bf72670d2 None None] MFA Rules not processed for user `ac8dfa5c97594e74a6839d91b292adcd`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:29.052119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-17a06be9-9b1d-4227-b12b-ab6bf72670d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:29.063732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533220c3-e4a9-4b71-84e5-ee9cf66556c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:29.077095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533220c3-e4a9-4b71-84e5-ee9cf66556c2 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:29.121868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 821/1657] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:29.123490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675efee-d461-4580-9f92-45532f4333c4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:29.123885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675efee-d461-4580-9f92-45532f4333c4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:29.124176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675efee-d461-4580-9f92-45532f4333c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:29.177879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0675efee-d461-4580-9f92-45532f4333c4 None None] MFA Rules not processed for user `9005cb23317a4a05b58e99284824135e`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:29.182207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0675efee-d461-4580-9f92-45532f4333c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:29.203753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 822/1658] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:29.205459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3ee4-7fbd-4080-ad4d-176baf9abf72 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:29.205771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3ee4-7fbd-4080-ad4d-176baf9abf72 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:29.206061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f3ee4-7fbd-4080-ad4d-176baf9abf72 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:29.211015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 823/1659] 10.222.0.98 () {54 vars in 912 bytes} [Fri Jul 15 15:53:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:53:29.215621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a6c30-e055-4326-b41b-51c8d9a75d91 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:29.215929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a6c30-e055-4326-b41b-51c8d9a75d91 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:29.218330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a6c30-e055-4326-b41b-51c8d9a75d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:29.263333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 837/1660] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:29.286033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-691a6c30-e055-4326-b41b-51c8d9a75d91 None None] MFA Rules not processed for user `9d20de9c921942d1afeb63e87272f782`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:29.299897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-691a6c30-e055-4326-b41b-51c8d9a75d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:29.317614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 824/1661] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:53:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:29.330236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a06be9-9b1d-4227-b12b-ab6bf72670d2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:29.384964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a06be9-9b1d-4227-b12b-ab6bf72670d2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:29.397572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26831832-887f-487d-bd54-8ed32a73fdd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:29.401035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831832-887f-487d-bd54-8ed32a73fdd3 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:29.401487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831832-887f-487d-bd54-8ed32a73fdd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:29.401776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831832-887f-487d-bd54-8ed32a73fdd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:29.431929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26831832-887f-487d-bd54-8ed32a73fdd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:29.433579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26831832-887f-487d-bd54-8ed32a73fdd3 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:29.469664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 838/1662] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:29.658095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7072ec-e746-4ad4-be49-24c6a5acbb84 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:29.658458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7072ec-e746-4ad4-be49-24c6a5acbb84 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:29.658676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7072ec-e746-4ad4-be49-24c6a5acbb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:29.725112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1c7072ec-e746-4ad4-be49-24c6a5acbb84 None None] MFA Rules not processed for user `78db558bb3ea4161b4a54e62f9ec3bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:29.727176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1c7072ec-e746-4ad4-be49-24c6a5acbb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:29.811053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:29.840778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 825/1663] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:30.445607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26831832-887f-487d-bd54-8ed32a73fdd3 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:30.471052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26831832-887f-487d-bd54-8ed32a73fdd3 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:30.480345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1b62f-fd13-48c7-be5d-279202141ddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:30.482235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1b62f-fd13-48c7-be5d-279202141ddf service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:30.483420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1b62f-fd13-48c7-be5d-279202141ddf service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:30.484321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1b62f-fd13-48c7-be5d-279202141ddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:30.496134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1b62f-fd13-48c7-be5d-279202141ddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:30.498156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1b62f-fd13-48c7-be5d-279202141ddf service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:30.540111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 839/1664] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:30.812851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7072ec-e746-4ad4-be49-24c6a5acbb84 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:30.841858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7072ec-e746-4ad4-be49-24c6a5acbb84 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:30.857858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21abe53-41a8-4f78-a461-459c3d739c81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:30.858844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21abe53-41a8-4f78-a461-459c3d739c81 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:30.859282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21abe53-41a8-4f78-a461-459c3d739c81 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:30.859779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21abe53-41a8-4f78-a461-459c3d739c81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:30.879830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21abe53-41a8-4f78-a461-459c3d739c81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:30.882612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21abe53-41a8-4f78-a461-459c3d739c81 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:30.938679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 826/1665] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:35.086558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9781e4-a480-4369-8498-73a70d427bdd None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:35.087119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9781e4-a480-4369-8498-73a70d427bdd None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:35.088046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9781e4-a480-4369-8498-73a70d427bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:35.223478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bc9781e4-a480-4369-8498-73a70d427bdd None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:35.225181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bc9781e4-a480-4369-8498-73a70d427bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:35.292529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21abe53-41a8-4f78-a461-459c3d739c81 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:35.514255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 840/1666] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:53:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:35.550766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21abe53-41a8-4f78-a461-459c3d739c81 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:35.555874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:35.557099 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:35.557605 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:35.558341 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] PATH_INFO: `/v3/users/a93503cfdcad4ec2a40a4865a31270c0` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:35.561077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a93503cfdcad4ec2a40a4865a31270c0)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:35.563886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:35.844366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a93503cfdcad4ec2a40a4865a31270c0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:35.866198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebeb70-981b-4038-9773-080f0052a478 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:35.866326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebeb70-981b-4038-9773-080f0052a478 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:35.867119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebeb70-981b-4038-9773-080f0052a478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:35.926821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5eebeb70-981b-4038-9773-080f0052a478 None None] MFA Rules not processed for user `ac8dfa5c97594e74a6839d91b292adcd`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:35.933777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a93503cfdcad4ec2a40a4865a31270c0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:35.934322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5eebeb70-981b-4038-9773-080f0052a478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:35.937704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a93503cfdcad4ec2a40a4865a31270c0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:35.949555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a93503cfdcad4ec2a40a4865a31270c0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:35.973703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 841/1667] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:35.977224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-31aa8014-b6a4-4d02-a621-d18343f00b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a93503cfdcad4ec2a40a4865a31270c0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:35.986847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebeb70-981b-4038-9773-080f0052a478 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:35.988679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5eebeb70-981b-4038-9773-080f0052a478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:36.062479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 827/1668] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:53:35 2022] DELETE /identity/v3/users/a93503cfdcad4ec2a40a4865a31270c0 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:53:36.067186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b9cc1-fc7c-456c-93e3-95fb544559e9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:36.067714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b9cc1-fc7c-456c-93e3-95fb544559e9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:36.069090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b9cc1-fc7c-456c-93e3-95fb544559e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:36.098888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:36.136929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-031b9cc1-fc7c-456c-93e3-95fb544559e9 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:36.138994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-031b9cc1-fc7c-456c-93e3-95fb544559e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:36.246262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebeb70-981b-4038-9773-080f0052a478 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:36.266019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:36.291544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 828/1669] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:36.295795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768bf4b-0101-4c82-a006-f35aff9c14db None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:36.296382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768bf4b-0101-4c82-a006-f35aff9c14db None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:36.296382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768bf4b-0101-4c82-a006-f35aff9c14db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:36.351787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c768bf4b-0101-4c82-a006-f35aff9c14db None None] MFA Rules not processed for user `78db558bb3ea4161b4a54e62f9ec3bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:36.355023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c768bf4b-0101-4c82-a006-f35aff9c14db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:36.436635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:36.437693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:36.438601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:36.439035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:36.442918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:36.513963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 829/1670] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:36.517083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768bf4b-0101-4c82-a006-f35aff9c14db None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:36.520053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c768bf4b-0101-4c82-a006-f35aff9c14db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:36.657457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:36.660711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:36.685377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768bf4b-0101-4c82-a006-f35aff9c14db None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:36.700440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 842/1671] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:53:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:53:36.733439 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b2f01-22a6-44ba-ba84-83bcfd9dab1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:36.734327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b2f01-22a6-44ba-ba84-83bcfd9dab1c service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:36.737604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b2f01-22a6-44ba-ba84-83bcfd9dab1c service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:36.738160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b2f01-22a6-44ba-ba84-83bcfd9dab1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:36.804266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b2f01-22a6-44ba-ba84-83bcfd9dab1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:36.808276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b2f01-22a6-44ba-ba84-83bcfd9dab1c service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:36.844274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 830/1672] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:37.886955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:37.889266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:37.921603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b574ff6-4e6e-49f1-a73f-9a3ebd7ed977 service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:37.937256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:37.938635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:37.939128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:37.939657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] PATH_INFO: `/v3/projects/5677770f291d42e493b4ef0dd346d049` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:37.948486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5677770f291d42e493b4ef0dd346d049)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:37.949665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:38.227487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:38.240562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5677770f291d42e493b4ef0dd346d049', 'request_id': 'req-ebc68cee-0eb2-44bf-9c7d-a061762f4727'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:38.247916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5677770f291d42e493b4ef0dd346d049', 'request_id': 'req-ebc68cee-0eb2-44bf-9c7d-a061762f4727'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:38.357198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5677770f291d42e493b4ef0dd346d049', 'request_id': 'req-ebc68cee-0eb2-44bf-9c7d-a061762f4727'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:38.373160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] The token cache is being invalidate because project 5677770f291d42e493b4ef0dd346d049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:53:38.373460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:38.374158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ebc68cee-0eb2-44bf-9c7d-a061762f4727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:38.381247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e220b0-e67d-480f-9913-df46b23cf0bd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:38.381748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e220b0-e67d-480f-9913-df46b23cf0bd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:38.381819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e220b0-e67d-480f-9913-df46b23cf0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:38.384164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 843/1673] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:53:37 2022] DELETE /identity/v3/projects/5677770f291d42e493b4ef0dd346d049 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:38.429051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-56e220b0-e67d-480f-9913-df46b23cf0bd None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:38.430650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-56e220b0-e67d-480f-9913-df46b23cf0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:38.563261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 831/1674] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:38.571070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba2f6d-8af1-42a7-b9ae-5090aa2e2837 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:38.571395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba2f6d-8af1-42a7-b9ae-5090aa2e2837 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:38.571700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba2f6d-8af1-42a7-b9ae-5090aa2e2837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:38.594075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-75ba2f6d-8af1-42a7-b9ae-5090aa2e2837 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:38.595762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-75ba2f6d-8af1-42a7-b9ae-5090aa2e2837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:38.604844 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 844/1675] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:38.612308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e220b0-e67d-480f-9913-df46b23cf0bd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:38.624359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e220b0-e67d-480f-9913-df46b23cf0bd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:38.631191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:38.632319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:38.632754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:38.633204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:38.634135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:38.634584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:38.636184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:53:38.636184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jul 15 15:53:38.636184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: Jul 15 15:53:38.637031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:38.654262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 832/1676] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:38.665419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba2f6d-8af1-42a7-b9ae-5090aa2e2837 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:38.676003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba2f6d-8af1-42a7-b9ae-5090aa2e2837 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:38.681111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81f7a5-ca3e-4ab1-8819-a2894e65940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:38.682088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81f7a5-ca3e-4ab1-8819-a2894e65940c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:38.682625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81f7a5-ca3e-4ab1-8819-a2894e65940c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:38.683217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81f7a5-ca3e-4ab1-8819-a2894e65940c admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:38.684218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81f7a5-ca3e-4ab1-8819-a2894e65940c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:38.686572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81f7a5-ca3e-4ab1-8819-a2894e65940c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:38.788070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 845/1677] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:38 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:38.788173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:38.799219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2193c-48f6-4a41-ab46-eaa9f0956d61 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:38.845597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:38.882318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde93ac0-8e67-40ac-bc72-4587b1cc6222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:38.883144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde93ac0-8e67-40ac-bc72-4587b1cc6222 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:38.885346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde93ac0-8e67-40ac-bc72-4587b1cc6222 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:38.885346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde93ac0-8e67-40ac-bc72-4587b1cc6222 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:38.885346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde93ac0-8e67-40ac-bc72-4587b1cc6222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:38.886704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde93ac0-8e67-40ac-bc72-4587b1cc6222 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:38.972323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 833/1678] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:38 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:53:38.977382 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81f7a5-ca3e-4ab1-8819-a2894e65940c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:38.991843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81f7a5-ca3e-4ab1-8819-a2894e65940c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:39.005668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5742486-2456-442b-a695-9f84a67aa9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:39.007062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5742486-2456-442b-a695-9f84a67aa9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:39.007840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5742486-2456-442b-a695-9f84a67aa9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:39.008261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5742486-2456-442b-a695-9f84a67aa9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:39.009128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5742486-2456-442b-a695-9f84a67aa9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:39.011202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5742486-2456-442b-a695-9f84a67aa9e0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:39.026572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 846/1679] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:39.031172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde93ac0-8e67-40ac-bc72-4587b1cc6222 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:39.043426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde93ac0-8e67-40ac-bc72-4587b1cc6222 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:39.049839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b693-d69b-4a7a-9846-d6e003fd842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:39.050285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b693-d69b-4a7a-9846-d6e003fd842c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:39.050553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b693-d69b-4a7a-9846-d6e003fd842c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:39.050836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b693-d69b-4a7a-9846-d6e003fd842c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:39.052293 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87b693-d69b-4a7a-9846-d6e003fd842c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:39.054428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87b693-d69b-4a7a-9846-d6e003fd842c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:39.071515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 834/1680] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:39.072518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5742486-2456-442b-a695-9f84a67aa9e0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:39.084037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5742486-2456-442b-a695-9f84a67aa9e0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:39.089399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3690f0d-b16b-480f-af9f-be055298f7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:39.091343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3690f0d-b16b-480f-af9f-be055298f7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:39.091864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3690f0d-b16b-480f-af9f-be055298f7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:39.094839 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3690f0d-b16b-480f-af9f-be055298f7cc admin admin] PATH_INFO: `/v3/projects/cb7682c725844eeaa1dd534cd74d3421/users/3a0064d22cee446792a5d2b65b6a9c06/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:39.141889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3690f0d-b16b-480f-af9f-be055298f7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7682c725844eeaa1dd534cd74d3421, user_id=3a0064d22cee446792a5d2b65b6a9c06, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:39.145934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3690f0d-b16b-480f-af9f-be055298f7cc admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:39.337659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 847/1681] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:39 2022] PUT /identity/v3/projects/cb7682c725844eeaa1dd534cd74d3421/users/3a0064d22cee446792a5d2b65b6a9c06/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:53:39.354968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ec758-0e04-4e50-ba3e-b5890d15f065 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:39.354968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ec758-0e04-4e50-ba3e-b5890d15f065 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:39.354968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ec758-0e04-4e50-ba3e-b5890d15f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:39.398464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-280ec758-0e04-4e50-ba3e-b5890d15f065 None None] MFA Rules not processed for user `3a0064d22cee446792a5d2b65b6a9c06`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:39.400788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-280ec758-0e04-4e50-ba3e-b5890d15f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:39.685095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 835/1682] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:39.690207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3690f0d-b16b-480f-af9f-be055298f7cc admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:39.790407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3690f0d-b16b-480f-af9f-be055298f7cc admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:39.807045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e96e79-f387-4ec5-a931-ccbc5d077930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:39.807045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e96e79-f387-4ec5-a931-ccbc5d077930 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:39.807045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e96e79-f387-4ec5-a931-ccbc5d077930 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:39.807045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e96e79-f387-4ec5-a931-ccbc5d077930 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:39.807660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e96e79-f387-4ec5-a931-ccbc5d077930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:39.810451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e96e79-f387-4ec5-a931-ccbc5d077930 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:39.956154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 848/1683] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:39 2022] POST /identity/v3/projects => generated 377 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:39.959825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ec758-0e04-4e50-ba3e-b5890d15f065 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:39.972626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ec758-0e04-4e50-ba3e-b5890d15f065 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:40.076537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11696d-cf83-4834-aafb-4a9e1a712aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:40.077726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11696d-cf83-4834-aafb-4a9e1a712aec admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:40.078167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11696d-cf83-4834-aafb-4a9e1a712aec admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:40.078445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11696d-cf83-4834-aafb-4a9e1a712aec admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:40.079494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11696d-cf83-4834-aafb-4a9e1a712aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:40.087715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11696d-cf83-4834-aafb-4a9e1a712aec admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:40.594067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 836/1684] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:39 2022] POST /identity/v3/users => generated 295 bytes in 635 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:53:40.598099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e96e79-f387-4ec5-a931-ccbc5d077930 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:40.610750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e96e79-f387-4ec5-a931-ccbc5d077930 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:40.616046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea4995-20e6-4471-bd3f-342095c8f493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:40.616737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea4995-20e6-4471-bd3f-342095c8f493 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:40.617028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea4995-20e6-4471-bd3f-342095c8f493 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:40.617307 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea4995-20e6-4471-bd3f-342095c8f493 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:40.618033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea4995-20e6-4471-bd3f-342095c8f493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:40.619725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea4995-20e6-4471-bd3f-342095c8f493 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:40.634053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 849/1685] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:40.637120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11696d-cf83-4834-aafb-4a9e1a712aec admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:40.654597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11696d-cf83-4834-aafb-4a9e1a712aec admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:40.662547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d546f14-0211-4b01-a833-33ae1402ba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:40.672233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d546f14-0211-4b01-a833-33ae1402ba6d admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:40.672905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d546f14-0211-4b01-a833-33ae1402ba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:40.673405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d546f14-0211-4b01-a833-33ae1402ba6d admin admin] PATH_INFO: `/v3/projects/af9239b3cc334441b79792dfd4308b6b/users/8c346779743c42b4931c96b10346745a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:40.736699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d546f14-0211-4b01-a833-33ae1402ba6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9239b3cc334441b79792dfd4308b6b, user_id=8c346779743c42b4931c96b10346745a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:40.739455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d546f14-0211-4b01-a833-33ae1402ba6d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:40.853098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 837/1686] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:40 2022] PUT /identity/v3/projects/af9239b3cc334441b79792dfd4308b6b/users/8c346779743c42b4931c96b10346745a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:40.853873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea4995-20e6-4471-bd3f-342095c8f493 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:40.871600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea4995-20e6-4471-bd3f-342095c8f493 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:40.984419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:41.018930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57721d3-38e9-4e55-ab5f-46db57d4e14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:41.020384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57721d3-38e9-4e55-ab5f-46db57d4e14e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:41.020861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57721d3-38e9-4e55-ab5f-46db57d4e14e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:41.021287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57721d3-38e9-4e55-ab5f-46db57d4e14e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:41.022216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57721d3-38e9-4e55-ab5f-46db57d4e14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:41.033965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57721d3-38e9-4e55-ab5f-46db57d4e14e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:41.040839 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 850/1687] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:41.044914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d546f14-0211-4b01-a833-33ae1402ba6d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:41.054911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:41.059073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d546f14-0211-4b01-a833-33ae1402ba6d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:41.068237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9de0c2-37c4-43e4-ae14-1c8ee65a3ccc None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:41.068903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9de0c2-37c4-43e4-ae14-1c8ee65a3ccc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:41.069409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9de0c2-37c4-43e4-ae14-1c8ee65a3ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:41.093337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3122add-02b7-47a2-9bf3-d4fea21d8a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:41.095253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3122add-02b7-47a2-9bf3-d4fea21d8a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:41.096926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3122add-02b7-47a2-9bf3-d4fea21d8a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:41.097224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3122add-02b7-47a2-9bf3-d4fea21d8a00 admin admin] PATH_INFO: `/v3/projects/af9239b3cc334441b79792dfd4308b6b/users/8c346779743c42b4931c96b10346745a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:41.104186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3122add-02b7-47a2-9bf3-d4fea21d8a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9239b3cc334441b79792dfd4308b6b, user_id=8c346779743c42b4931c96b10346745a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:41.104833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1b9de0c2-37c4-43e4-ae14-1c8ee65a3ccc None None] MFA Rules not processed for user `9d20de9c921942d1afeb63e87272f782`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:41.106910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1b9de0c2-37c4-43e4-ae14-1c8ee65a3ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:41.123897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3122add-02b7-47a2-9bf3-d4fea21d8a00 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:41.197650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 838/1688] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:41 2022] PUT /identity/v3/projects/af9239b3cc334441b79792dfd4308b6b/users/8c346779743c42b4931c96b10346745a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:41.200825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500caf4c-45be-4b27-9857-2c7a51cba23e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:41.201446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500caf4c-45be-4b27-9857-2c7a51cba23e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:41.203352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500caf4c-45be-4b27-9857-2c7a51cba23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:41.248472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 851/1689] 10.222.0.75 () {58 vars in 1015 bytes} [Fri Jul 15 15:53:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:41.259830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-500caf4c-45be-4b27-9857-2c7a51cba23e None None] MFA Rules not processed for user `8c346779743c42b4931c96b10346745a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:41.261469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-500caf4c-45be-4b27-9857-2c7a51cba23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:41.371364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 839/1690] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:41.410454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9de0c2-37c4-43e4-ae14-1c8ee65a3ccc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:41.414468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1b9de0c2-37c4-43e4-ae14-1c8ee65a3ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:41.443855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500caf4c-45be-4b27-9857-2c7a51cba23e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:41.445512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-500caf4c-45be-4b27-9857-2c7a51cba23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:41.642111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9de0c2-37c4-43e4-ae14-1c8ee65a3ccc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:41.655712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500caf4c-45be-4b27-9857-2c7a51cba23e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:41.667609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecb437-5c73-42aa-b2b3-b86d31e10e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:41.669230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecb437-5c73-42aa-b2b3-b86d31e10e09 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:41.670042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecb437-5c73-42aa-b2b3-b86d31e10e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:41.670690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecb437-5c73-42aa-b2b3-b86d31e10e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:41.691497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecb437-5c73-42aa-b2b3-b86d31e10e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:41.694935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ecb437-5c73-42aa-b2b3-b86d31e10e09 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:41.728797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 840/1691] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:41.740854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecb437-5c73-42aa-b2b3-b86d31e10e09 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:41.766566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecb437-5c73-42aa-b2b3-b86d31e10e09 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:41.779488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36786b-2f65-453e-955f-ec779d447a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:41.780909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36786b-2f65-453e-955f-ec779d447a0d service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:41.781417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36786b-2f65-453e-955f-ec779d447a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:41.783044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36786b-2f65-453e-955f-ec779d447a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:41.800273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98943e-2e8d-45a0-a77b-64407a3cc8b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:41.801064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98943e-2e8d-45a0-a77b-64407a3cc8b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:41.801356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98943e-2e8d-45a0-a77b-64407a3cc8b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:41.803729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98943e-2e8d-45a0-a77b-64407a3cc8b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:41.808345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36786b-2f65-453e-955f-ec779d447a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:41.813321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36786b-2f65-453e-955f-ec779d447a0d service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:41.857534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 841/1692] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:41.861495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36786b-2f65-453e-955f-ec779d447a0d service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:41.892558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36786b-2f65-453e-955f-ec779d447a0d service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:41.902033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:41.903481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:41.904060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:41.904546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:41.913951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:41.915933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:41.926735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 842/1693] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:41.984041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98943e-2e8d-45a0-a77b-64407a3cc8b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:41.984041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98943e-2e8d-45a0-a77b-64407a3cc8b0 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:41.989715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:42.002534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 852/1694] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:42.838312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:42.848853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:43.014594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:43.045498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a09d4e-89a0-43e3-878c-dbee0427a80b service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:43.052266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:43.053948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:43.054587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:43.055488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] PATH_INFO: `/v3/users/2ff316f85f3e40178a0e7354a123e483` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:43.057772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ff316f85f3e40178a0e7354a123e483)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:43.063271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:43.397501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ff316f85f3e40178a0e7354a123e483'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:43.435326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ff316f85f3e40178a0e7354a123e483'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:43.438897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ff316f85f3e40178a0e7354a123e483'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:43.441075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ff316f85f3e40178a0e7354a123e483'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:43.473163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ff316f85f3e40178a0e7354a123e483'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:43.623916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 843/1695] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:53:42 2022] DELETE /identity/v3/users/2ff316f85f3e40178a0e7354a123e483 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:43.628503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74dfec-0835-4a40-b63d-7d10345856de None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:43.629830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74dfec-0835-4a40-b63d-7d10345856de None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:43.630731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74dfec-0835-4a40-b63d-7d10345856de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:43.675381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-be74dfec-0835-4a40-b63d-7d10345856de None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:43.679613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-be74dfec-0835-4a40-b63d-7d10345856de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:43.808455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 853/1696] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:43.819329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:43.821182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:43.928693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b43113-c89b-4381-852b-0b710c5784d8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:43.932607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240c4e2-aa06-4b47-8c28-a52ff34bab6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:43.933256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c4e2-aa06-4b47-8c28-a52ff34bab6b service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:43.933544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c4e2-aa06-4b47-8c28-a52ff34bab6b service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:43.933852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c4e2-aa06-4b47-8c28-a52ff34bab6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:43.962676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240c4e2-aa06-4b47-8c28-a52ff34bab6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:43.964698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240c4e2-aa06-4b47-8c28-a52ff34bab6b service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:44.002186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 844/1697] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:44.960581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74dfec-0835-4a40-b63d-7d10345856de None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:44.963379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-be74dfec-0835-4a40-b63d-7d10345856de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:45.004947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74dfec-0835-4a40-b63d-7d10345856de None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:45.014184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:45.014184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:45.014184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:45.031747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] PATH_INFO: `/v3/projects/c5b5c3ebb2ec42988f6203ec34392dc6` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:45.044170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b5c3ebb2ec42988f6203ec34392dc6)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:45.055022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:45.284359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b5c3ebb2ec42988f6203ec34392dc6', 'request_id': 'req-88875ccc-ce05-4499-a2be-7f174af15e2b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:45.309174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b5c3ebb2ec42988f6203ec34392dc6', 'request_id': 'req-88875ccc-ce05-4499-a2be-7f174af15e2b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:45.352625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fdefc-7730-4c31-8d40-d8733f501b5a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:45.353212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fdefc-7730-4c31-8d40-d8733f501b5a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:45.353212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fdefc-7730-4c31-8d40-d8733f501b5a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:45.354585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 845/1698] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:53:45.373074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b5c3ebb2ec42988f6203ec34392dc6', 'request_id': 'req-88875ccc-ce05-4499-a2be-7f174af15e2b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:45.382149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca14a9-850b-4aed-9673-9c5c3f1a1fcf None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:45.382706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca14a9-850b-4aed-9673-9c5c3f1a1fcf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:45.383138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca14a9-850b-4aed-9673-9c5c3f1a1fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:45.420216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-58ca14a9-850b-4aed-9673-9c5c3f1a1fcf None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:45.421455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-58ca14a9-850b-4aed-9673-9c5c3f1a1fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:45.424329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] The token cache is being invalidate because project c5b5c3ebb2ec42988f6203ec34392dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:53:45.424900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:45.425700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88875ccc-ce05-4499-a2be-7f174af15e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:45.433650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 854/1699] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:53:44 2022] DELETE /identity/v3/projects/c5b5c3ebb2ec42988f6203ec34392dc6 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:45.442481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b12ef-f4bb-45a3-b746-4469f8bfea80 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:45.442481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b12ef-f4bb-45a3-b746-4469f8bfea80 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:45.442481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b12ef-f4bb-45a3-b746-4469f8bfea80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:45.517141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7e1b12ef-f4bb-45a3-b746-4469f8bfea80 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:45.520869 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7e1b12ef-f4bb-45a3-b746-4469f8bfea80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:45.660069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 846/1700] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:53:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:53:45.699160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca14a9-850b-4aed-9673-9c5c3f1a1fcf None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:45.701511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-58ca14a9-850b-4aed-9673-9c5c3f1a1fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:45.705860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 855/1701] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:45.714158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e1569-c299-4ebf-a9b5-c2b753481d10 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:45.714834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e1569-c299-4ebf-a9b5-c2b753481d10 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:45.715278 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e1569-c299-4ebf-a9b5-c2b753481d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:45.740156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca14a9-850b-4aed-9673-9c5c3f1a1fcf None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:45.765191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6862de2-66ca-426a-ace7-d076038214f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:45.768203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862de2-66ca-426a-ace7-d076038214f5 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:45.768762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862de2-66ca-426a-ace7-d076038214f5 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:45.769377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6862de2-66ca-426a-ace7-d076038214f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:45.790408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:45.792047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1e1e1569-c299-4ebf-a9b5-c2b753481d10 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:45.795325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1e1e1569-c299-4ebf-a9b5-c2b753481d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:45.802556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6862de2-66ca-426a-ace7-d076038214f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:45.806170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6862de2-66ca-426a-ace7-d076038214f5 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:45.878247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 856/1702] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:45.882117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e1569-c299-4ebf-a9b5-c2b753481d10 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:45.887164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 847/1703] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:53:45.910878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e1569-c299-4ebf-a9b5-c2b753481d10 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:45.920865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:45.921613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:45.921915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:45.922348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:45.923093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:45.923524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:45.925301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:45.949859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 857/1704] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:45.955281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6862de2-66ca-426a-ace7-d076038214f5 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:46.025990 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6862de2-66ca-426a-ace7-d076038214f5 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:46.068320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946416-8730-40b9-93e8-450f2f52a44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:46.068320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946416-8730-40b9-93e8-450f2f52a44f admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:46.068320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946416-8730-40b9-93e8-450f2f52a44f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:46.068320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946416-8730-40b9-93e8-450f2f52a44f admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:46.068658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1946416-8730-40b9-93e8-450f2f52a44f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:46.076111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1946416-8730-40b9-93e8-450f2f52a44f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:46.197283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 848/1705] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:45 2022] POST /identity/v3/projects => generated 365 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:46.198563 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:46.208384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:46.214822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5a8f0-a1af-4e0f-ba2c-115c920bb992 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:46.313218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d30a1-da1c-4897-98f2-009f24fd842a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:46.314350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d30a1-da1c-4897-98f2-009f24fd842a admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:46.314938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d30a1-da1c-4897-98f2-009f24fd842a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:46.315842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d30a1-da1c-4897-98f2-009f24fd842a admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:46.317153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d30a1-da1c-4897-98f2-009f24fd842a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:46.321007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d30a1-da1c-4897-98f2-009f24fd842a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:46.420406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 858/1706] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:46 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:46.424913 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946416-8730-40b9-93e8-450f2f52a44f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:46.443502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946416-8730-40b9-93e8-450f2f52a44f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:46.455483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151704-9006-428c-8b23-c093d0654e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:46.458311 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151704-9006-428c-8b23-c093d0654e8d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:46.458788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151704-9006-428c-8b23-c093d0654e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:46.459238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151704-9006-428c-8b23-c093d0654e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:46.460135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84151704-9006-428c-8b23-c093d0654e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:46.470241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84151704-9006-428c-8b23-c093d0654e8d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:46.504845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 849/1707] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:46.508855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d30a1-da1c-4897-98f2-009f24fd842a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:46.534012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d30a1-da1c-4897-98f2-009f24fd842a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:46.558337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cde31-5c3a-4cca-9d54-ef2b09416443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:46.559717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cde31-5c3a-4cca-9d54-ef2b09416443 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:46.560659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cde31-5c3a-4cca-9d54-ef2b09416443 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:46.561189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cde31-5c3a-4cca-9d54-ef2b09416443 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:46.562904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cde31-5c3a-4cca-9d54-ef2b09416443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:46.567030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cde31-5c3a-4cca-9d54-ef2b09416443 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:46.598755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 859/1708] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:46.599426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151704-9006-428c-8b23-c093d0654e8d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:46.626010 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151704-9006-428c-8b23-c093d0654e8d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:46.643965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:46.650796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:46.651407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:46.652183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] PATH_INFO: `/v3/projects/61d33542b62342ddb514e4b35c6e94b0/users/7b4b2714c6cf4d3885843766d24d3485/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:46.691483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d33542b62342ddb514e4b35c6e94b0, user_id=7b4b2714c6cf4d3885843766d24d3485, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:46.698575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:46.774910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 850/1709] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:46 2022] PUT /identity/v3/projects/61d33542b62342ddb514e4b35c6e94b0/users/7b4b2714c6cf4d3885843766d24d3485/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:46.787024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8211-940d-4ecb-9dbb-38aeaad31a4b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:46.787663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8211-940d-4ecb-9dbb-38aeaad31a4b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:46.788169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8211-940d-4ecb-9dbb-38aeaad31a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:46.893229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8a0b8211-940d-4ecb-9dbb-38aeaad31a4b None None] MFA Rules not processed for user `7b4b2714c6cf4d3885843766d24d3485`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:46.894879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8a0b8211-940d-4ecb-9dbb-38aeaad31a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:47.063123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 860/1710] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:47.097047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:47.099411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:47.245148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8211-940d-4ecb-9dbb-38aeaad31a4b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:47.248379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9105905-ff02-4471-8ab2-e02240913397 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:47.248520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8a0b8211-940d-4ecb-9dbb-38aeaad31a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:47.258767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3cdf-c78b-40aa-9ced-9be12b43b1a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:47.259752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3cdf-c78b-40aa-9ced-9be12b43b1a2 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:47.261060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3cdf-c78b-40aa-9ced-9be12b43b1a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:47.261514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3cdf-c78b-40aa-9ced-9be12b43b1a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:47.291906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb3cdf-c78b-40aa-9ced-9be12b43b1a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:47.293939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb3cdf-c78b-40aa-9ced-9be12b43b1a2 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:47.312521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 851/1711] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:47.339944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8211-940d-4ecb-9dbb-38aeaad31a4b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:47.346241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:47.351577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:47.351577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:47.351577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] PATH_INFO: `/v3/users/d03681f0b895409e847b3a40bf43ac45` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:47.351577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03681f0b895409e847b3a40bf43ac45)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:47.351577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:53:47.351577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jul 15 15:53:47.351577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: Jul 15 15:53:47.352112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:47.377160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:47.655972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03681f0b895409e847b3a40bf43ac45'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:47.700911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03681f0b895409e847b3a40bf43ac45'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:47.701872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03681f0b895409e847b3a40bf43ac45'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:47.703161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03681f0b895409e847b3a40bf43ac45'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:47.713623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03681f0b895409e847b3a40bf43ac45'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:47.809324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 861/1712] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:53:47 2022] DELETE /identity/v3/users/d03681f0b895409e847b3a40bf43ac45 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:47.813984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d57edd-c8e4-48e5-99ab-01ebed26de6a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:47.818842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d57edd-c8e4-48e5-99ab-01ebed26de6a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:47.818842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d57edd-c8e4-48e5-99ab-01ebed26de6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:47.831337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-14d57edd-c8e4-48e5-99ab-01ebed26de6a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:47.832707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-14d57edd-c8e4-48e5-99ab-01ebed26de6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:47.892919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 852/1713] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:47.902886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:47.905148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:48.059945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c906-ea66-4d6e-89eb-ed04630b69b9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:48.064228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d61aa5-90a4-4551-a84c-fb2fd314b6ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:48.064996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d61aa5-90a4-4551-a84c-fb2fd314b6ca service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:48.065282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d61aa5-90a4-4551-a84c-fb2fd314b6ca service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:48.065560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d61aa5-90a4-4551-a84c-fb2fd314b6ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:48.081090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d61aa5-90a4-4551-a84c-fb2fd314b6ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:48.087754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d61aa5-90a4-4551-a84c-fb2fd314b6ca service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:48.114785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 862/1714] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:53:48.770811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d57edd-c8e4-48e5-99ab-01ebed26de6a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:48.772321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-14d57edd-c8e4-48e5-99ab-01ebed26de6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:48.785260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d57edd-c8e4-48e5-99ab-01ebed26de6a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:48.789841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:48.790853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:48.791388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:48.791670 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] PATH_INFO: `/v3/projects/c02b7fdd2ebe442cad31bc5d84fb7ba5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:48.793049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02b7fdd2ebe442cad31bc5d84fb7ba5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:48.794149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:53:48.794149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jul 15 15:53:48.794149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: Jul 15 15:53:48.795519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:48.896865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:48.913189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02b7fdd2ebe442cad31bc5d84fb7ba5', 'request_id': 'req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:48.928048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02b7fdd2ebe442cad31bc5d84fb7ba5', 'request_id': 'req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:48.939319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02b7fdd2ebe442cad31bc5d84fb7ba5', 'request_id': 'req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:49.071629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] The token cache is being invalidate because project c02b7fdd2ebe442cad31bc5d84fb7ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:53:49.071922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:49.072549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:49.074117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 853/1715] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:53:48 2022] DELETE /identity/v3/projects/c02b7fdd2ebe442cad31bc5d84fb7ba5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:49.078202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d61aa5-90a4-4551-a84c-fb2fd314b6ca service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:49.088355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d61aa5-90a4-4551-a84c-fb2fd314b6ca service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:49.144880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:49.145749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:49.146040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:49.146325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] PATH_INFO: `/v3/users/0e7fc7233b2a4f928a4151aa2799f163` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:49.147621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e7fc7233b2a4f928a4151aa2799f163)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:49.149480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:49.447219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:49.456049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e7fc7233b2a4f928a4151aa2799f163'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:49.468737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e7fc7233b2a4f928a4151aa2799f163'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:49.469416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e7fc7233b2a4f928a4151aa2799f163'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:49.470019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e7fc7233b2a4f928a4151aa2799f163'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:49.476808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d942c683-7479-490a-9f4f-54bc4f29bd07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e7fc7233b2a4f928a4151aa2799f163'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:49.567284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 863/1716] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:53:49 2022] DELETE /identity/v3/users/0e7fc7233b2a4f928a4151aa2799f163 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:50.108180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:50.110346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:50.125706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc2e21-cfa7-4c04-8e5f-b0185e6c3d29 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:50.193303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:50.194186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:50.194472 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:50.194749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] PATH_INFO: `/v3/projects/7af5872b9b0c47a1a0acc541624d5a06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:50.196066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af5872b9b0c47a1a0acc541624d5a06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:50.198031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:50.353770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af5872b9b0c47a1a0acc541624d5a06', 'request_id': 'req-8973fb6d-b896-41e3-9489-44cbd5d2691f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:50.360792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af5872b9b0c47a1a0acc541624d5a06', 'request_id': 'req-8973fb6d-b896-41e3-9489-44cbd5d2691f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:50.366811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af5872b9b0c47a1a0acc541624d5a06', 'request_id': 'req-8973fb6d-b896-41e3-9489-44cbd5d2691f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:50.433027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] The token cache is being invalidate because project 7af5872b9b0c47a1a0acc541624d5a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:53:50.433319 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:50.434057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8973fb6d-b896-41e3-9489-44cbd5d2691f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:50.439054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 854/1717] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:53:50 2022] DELETE /identity/v3/projects/7af5872b9b0c47a1a0acc541624d5a06 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:50.441198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccab95-a0b8-4019-a339-e27b54f0c7ba None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:50.441670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccab95-a0b8-4019-a339-e27b54f0c7ba None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:50.441766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccab95-a0b8-4019-a339-e27b54f0c7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:50.485182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d3ccab95-a0b8-4019-a339-e27b54f0c7ba None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:50.486438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d3ccab95-a0b8-4019-a339-e27b54f0c7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:50.599308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 864/1718] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:50.609092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a78d-b3bc-4c25-8f1b-785141128877 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:50.609382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a78d-b3bc-4c25-8f1b-785141128877 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:50.609697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119a78d-b3bc-4c25-8f1b-785141128877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:50.645941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f119a78d-b3bc-4c25-8f1b-785141128877 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:50.647415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f119a78d-b3bc-4c25-8f1b-785141128877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:50.661210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 855/1719] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:50.662282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccab95-a0b8-4019-a339-e27b54f0c7ba None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:50.675573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccab95-a0b8-4019-a339-e27b54f0c7ba None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:50.681011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:50.681927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:50.682397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:50.682881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:50.683840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:50.685316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:50.689798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:50.706396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 865/1720] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:50.707757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119a78d-b3bc-4c25-8f1b-785141128877 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:50.725416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119a78d-b3bc-4c25-8f1b-785141128877 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:50.729955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41a56-1886-41dd-b5dd-15720512c0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:50.731621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41a56-1886-41dd-b5dd-15720512c0dc admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:50.732094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41a56-1886-41dd-b5dd-15720512c0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:50.732569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41a56-1886-41dd-b5dd-15720512c0dc admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:50.733523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e41a56-1886-41dd-b5dd-15720512c0dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:50.735785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e41a56-1886-41dd-b5dd-15720512c0dc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:50.864777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 856/1721] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:50 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:50.866345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:50.876873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e9423-8c82-4104-8dda-b120adbd5efb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:50.939043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910702-d93f-4343-8049-b91ae057fe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:50.939830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910702-d93f-4343-8049-b91ae057fe9c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:50.940259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910702-d93f-4343-8049-b91ae057fe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:50.940521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11910702-d93f-4343-8049-b91ae057fe9c admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:50.941283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11910702-d93f-4343-8049-b91ae057fe9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:50.945667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11910702-d93f-4343-8049-b91ae057fe9c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:51.062847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 866/1722] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:50 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:51.064140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41a56-1886-41dd-b5dd-15720512c0dc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:51.074419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41a56-1886-41dd-b5dd-15720512c0dc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:51.079547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d744d-e8a3-4af8-8ade-2dafc5cad81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:51.080260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d744d-e8a3-4af8-8ade-2dafc5cad81e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:51.080547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d744d-e8a3-4af8-8ade-2dafc5cad81e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:51.080819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d744d-e8a3-4af8-8ade-2dafc5cad81e admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:51.081542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d744d-e8a3-4af8-8ade-2dafc5cad81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:51.083137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d744d-e8a3-4af8-8ade-2dafc5cad81e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:51.095833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 857/1723] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:51.099963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910702-d93f-4343-8049-b91ae057fe9c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:51.109911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11910702-d93f-4343-8049-b91ae057fe9c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:51.114308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b2e63-a01b-46ff-b9d3-61c48ad3d97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:51.115003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b2e63-a01b-46ff-b9d3-61c48ad3d97f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:51.115285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b2e63-a01b-46ff-b9d3-61c48ad3d97f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:51.115592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b2e63-a01b-46ff-b9d3-61c48ad3d97f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:51.116304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b2e63-a01b-46ff-b9d3-61c48ad3d97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:51.117903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b2e63-a01b-46ff-b9d3-61c48ad3d97f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:51.128504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 867/1724] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:51.132458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d744d-e8a3-4af8-8ade-2dafc5cad81e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:51.143329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d744d-e8a3-4af8-8ade-2dafc5cad81e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:51.147872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc156d1-314d-441e-910a-e40a7b83a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:51.148739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc156d1-314d-441e-910a-e40a7b83a30d admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:51.149024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc156d1-314d-441e-910a-e40a7b83a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:51.149300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc156d1-314d-441e-910a-e40a7b83a30d admin admin] PATH_INFO: `/v3/projects/748d4c63887e46359eca571dc9bbb1f1/users/59aac99258fb457fb8129a302d914347/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:51.162840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:51.166674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc156d1-314d-441e-910a-e40a7b83a30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=748d4c63887e46359eca571dc9bbb1f1, user_id=59aac99258fb457fb8129a302d914347, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:51.169551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc156d1-314d-441e-910a-e40a7b83a30d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:51.270885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 858/1725] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:51 2022] PUT /identity/v3/projects/748d4c63887e46359eca571dc9bbb1f1/users/59aac99258fb457fb8129a302d914347/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:51.281344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376e56d-a756-4140-9c97-d3548fdc3140 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:51.281754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376e56d-a756-4140-9c97-d3548fdc3140 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:51.282103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376e56d-a756-4140-9c97-d3548fdc3140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:51.311254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5376e56d-a756-4140-9c97-d3548fdc3140 None None] MFA Rules not processed for user `59aac99258fb457fb8129a302d914347`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:51.314098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5376e56d-a756-4140-9c97-d3548fdc3140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:51.323522 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:51.397852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 868/1726] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:51.398507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc156d1-314d-441e-910a-e40a7b83a30d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:51.461961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc156d1-314d-441e-910a-e40a7b83a30d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:51.467554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206c0e-6176-4e73-9c61-c21f3a15967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:51.468232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206c0e-6176-4e73-9c61-c21f3a15967a admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:51.468524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206c0e-6176-4e73-9c61-c21f3a15967a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:51.468809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9206c0e-6176-4e73-9c61-c21f3a15967a admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:51.469513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9206c0e-6176-4e73-9c61-c21f3a15967a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:51.471477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9206c0e-6176-4e73-9c61-c21f3a15967a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:51.604684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 859/1727] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:51 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:51.608814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376e56d-a756-4140-9c97-d3548fdc3140 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:51.620230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376e56d-a756-4140-9c97-d3548fdc3140 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:51.680772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105573ec-bd42-4015-906d-be8df53e977c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:51.681553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105573ec-bd42-4015-906d-be8df53e977c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:51.681842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105573ec-bd42-4015-906d-be8df53e977c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:51.682123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105573ec-bd42-4015-906d-be8df53e977c admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:51.682853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105573ec-bd42-4015-906d-be8df53e977c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:51.684906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105573ec-bd42-4015-906d-be8df53e977c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:51.767017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 869/1728] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:51 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:53:51.769209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206c0e-6176-4e73-9c61-c21f3a15967a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:51.784680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9206c0e-6176-4e73-9c61-c21f3a15967a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:51.789222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b660ef-4d52-4dd9-b186-933a4063847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:51.789885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b660ef-4d52-4dd9-b186-933a4063847f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:51.790169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b660ef-4d52-4dd9-b186-933a4063847f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:51.790573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b660ef-4d52-4dd9-b186-933a4063847f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:51.791318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b660ef-4d52-4dd9-b186-933a4063847f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:51.792939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b660ef-4d52-4dd9-b186-933a4063847f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:51.804103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 860/1729] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:51.808158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105573ec-bd42-4015-906d-be8df53e977c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:51.818206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105573ec-bd42-4015-906d-be8df53e977c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:51.822851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17a0ca-2a6a-4799-a08d-d7ff484f1c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:51.823769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a0ca-2a6a-4799-a08d-d7ff484f1c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:51.824058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a0ca-2a6a-4799-a08d-d7ff484f1c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:51.824339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17a0ca-2a6a-4799-a08d-d7ff484f1c81 admin admin] PATH_INFO: `/v3/projects/ab11d7af982341ae894f3661f9f9f8f7/users/c9a9725fe46545cb8882f8f826192311/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:51.840564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17a0ca-2a6a-4799-a08d-d7ff484f1c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab11d7af982341ae894f3661f9f9f8f7, user_id=c9a9725fe46545cb8882f8f826192311, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:51.843765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17a0ca-2a6a-4799-a08d-d7ff484f1c81 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:51.958177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 870/1730] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:51 2022] PUT /identity/v3/projects/ab11d7af982341ae894f3661f9f9f8f7/users/c9a9725fe46545cb8882f8f826192311/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:51.961916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b660ef-4d52-4dd9-b186-933a4063847f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:51.973283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b660ef-4d52-4dd9-b186-933a4063847f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:52.032443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a95dc-728e-4b6c-9c15-2bdb297bcd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:52.033123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a95dc-728e-4b6c-9c15-2bdb297bcd1b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:52.033406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a95dc-728e-4b6c-9c15-2bdb297bcd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:52.033679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a95dc-728e-4b6c-9c15-2bdb297bcd1b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:52.034395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a95dc-728e-4b6c-9c15-2bdb297bcd1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:52.036162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a95dc-728e-4b6c-9c15-2bdb297bcd1b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:52.047215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 861/1731] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:52.051858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17a0ca-2a6a-4799-a08d-d7ff484f1c81 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:52.063008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17a0ca-2a6a-4799-a08d-d7ff484f1c81 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:52.068713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:52.070023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:52.070407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:52.070752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] PATH_INFO: `/v3/projects/ab11d7af982341ae894f3661f9f9f8f7/users/c9a9725fe46545cb8882f8f826192311/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:52.072898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab11d7af982341ae894f3661f9f9f8f7, user_id=c9a9725fe46545cb8882f8f826192311, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:52.078010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:52.088132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 871/1732] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:52 2022] PUT /identity/v3/projects/ab11d7af982341ae894f3661f9f9f8f7/users/c9a9725fe46545cb8882f8f826192311/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:52.100005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015981-bbb0-4ff1-95b4-7b67c72dda4e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:52.100486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015981-bbb0-4ff1-95b4-7b67c72dda4e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:52.100874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015981-bbb0-4ff1-95b4-7b67c72dda4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:52.131838 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-42015981-bbb0-4ff1-95b4-7b67c72dda4e None None] MFA Rules not processed for user `c9a9725fe46545cb8882f8f826192311`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:52.133064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-42015981-bbb0-4ff1-95b4-7b67c72dda4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:52.183746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:52.207646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 862/1733] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:53:52.219225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:52.221099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:52.280373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b82d5-d8ad-4f4b-9fcc-98adeaef7bd5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:52.283981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96b3b-f09b-4eb1-8984-be64be4eb07c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:52.284629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96b3b-f09b-4eb1-8984-be64be4eb07c service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:52.284914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96b3b-f09b-4eb1-8984-be64be4eb07c service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:52.285189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96b3b-f09b-4eb1-8984-be64be4eb07c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:52.294597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96b3b-f09b-4eb1-8984-be64be4eb07c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:52.296056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96b3b-f09b-4eb1-8984-be64be4eb07c service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:52.305532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 872/1734] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:52.334404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42015981-bbb0-4ff1-95b4-7b67c72dda4e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:52.416684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42015981-bbb0-4ff1-95b4-7b67c72dda4e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:52.421816 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:52.422693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:52.422981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:52.423265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] PATH_INFO: `/v3/users/59aac99258fb457fb8129a302d914347` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:52.424488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59aac99258fb457fb8129a302d914347)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:52.426444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:52.617142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59aac99258fb457fb8129a302d914347'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:52.629385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59aac99258fb457fb8129a302d914347'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:52.630059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59aac99258fb457fb8129a302d914347'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:52.630684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59aac99258fb457fb8129a302d914347'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:52.637532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59aac99258fb457fb8129a302d914347'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:52.719889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 863/1735] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:53:52 2022] DELETE /identity/v3/users/59aac99258fb457fb8129a302d914347 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:52.727558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6264-65f8-498e-bcdd-ea3552abc290 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:52.728001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6264-65f8-498e-bcdd-ea3552abc290 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:52.728102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6264-65f8-498e-bcdd-ea3552abc290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:52.735460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:52.745755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-febc6264-65f8-498e-bcdd-ea3552abc290 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:52.746950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-febc6264-65f8-498e-bcdd-ea3552abc290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:52.809778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 873/1736] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:52.819687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:52.821189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:52.880057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994e613-9f70-400f-8021-a0f91d87b612 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:52.884806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f4c58-ff81-4038-bf32-17aeb7e24402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:52.884806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f4c58-ff81-4038-bf32-17aeb7e24402 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:52.884994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f4c58-ff81-4038-bf32-17aeb7e24402 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:52.885276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f4c58-ff81-4038-bf32-17aeb7e24402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:52.894811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f4c58-ff81-4038-bf32-17aeb7e24402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:52.896324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f4c58-ff81-4038-bf32-17aeb7e24402 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:52.906637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 864/1737] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:53.311755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc6264-65f8-498e-bcdd-ea3552abc290 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:53.313303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-febc6264-65f8-498e-bcdd-ea3552abc290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:53.327927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc6264-65f8-498e-bcdd-ea3552abc290 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:53.334648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:53.335648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:53.335941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:53.336272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] PATH_INFO: `/v3/projects/748d4c63887e46359eca571dc9bbb1f1` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:53.337905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=748d4c63887e46359eca571dc9bbb1f1)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:53.340369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:53.468438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '748d4c63887e46359eca571dc9bbb1f1', 'request_id': 'req-b568427a-a534-483e-85e3-27f373e86b3e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:53.479161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '748d4c63887e46359eca571dc9bbb1f1', 'request_id': 'req-b568427a-a534-483e-85e3-27f373e86b3e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:53.623038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '748d4c63887e46359eca571dc9bbb1f1', 'request_id': 'req-b568427a-a534-483e-85e3-27f373e86b3e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:53.645064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] The token cache is being invalidate because project 748d4c63887e46359eca571dc9bbb1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:53:53.645511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:53.646440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:53.651278 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 874/1738] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:53:53 2022] DELETE /identity/v3/projects/748d4c63887e46359eca571dc9bbb1f1 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:53:53.652460 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f4c58-ff81-4038-bf32-17aeb7e24402 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:53.665626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f4c58-ff81-4038-bf32-17aeb7e24402 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:53.671525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:53.733744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:53.734723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:53.735147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:53.735551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] PATH_INFO: `/v3/users/c9a9725fe46545cb8882f8f826192311` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:53.736897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a9725fe46545cb8882f8f826192311)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:53.739234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:53.846087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:53.847966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:53.918207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568427a-a534-483e-85e3-27f373e86b3e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:53.923957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:53.925062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:53.925353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:53.925630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:53.927854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:53.958755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a9725fe46545cb8882f8f826192311'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:53.995588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a9725fe46545cb8882f8f826192311'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:53.998264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:53.999591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a9725fe46545cb8882f8f826192311'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:53.999591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a9725fe46545cb8882f8f826192311'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:54.010593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e1c1d82-2881-4e29-a87b-132a522828fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a9725fe46545cb8882f8f826192311'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:54.037331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:54.038823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:54.062734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 875/1739] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:53:54.090509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 865/1740] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:53:53 2022] DELETE /identity/v3/users/c9a9725fe46545cb8882f8f826192311 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:53:55.131721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:55.133360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:55.155050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8f87-ab23-4728-b3a3-78d821fe4f20 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:55.239863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:55.240886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:55.241255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:55.241615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] PATH_INFO: `/v3/projects/ab11d7af982341ae894f3661f9f9f8f7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:55.242979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab11d7af982341ae894f3661f9f9f8f7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:55.245518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:55.417903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab11d7af982341ae894f3661f9f9f8f7', 'request_id': 'req-b3669f03-6d7a-4593-9321-df5a0840b48a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:55.425879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab11d7af982341ae894f3661f9f9f8f7', 'request_id': 'req-b3669f03-6d7a-4593-9321-df5a0840b48a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:55.501879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab11d7af982341ae894f3661f9f9f8f7', 'request_id': 'req-b3669f03-6d7a-4593-9321-df5a0840b48a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:55.528081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] The token cache is being invalidate because project ab11d7af982341ae894f3661f9f9f8f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:53:55.528459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:55.529860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b3669f03-6d7a-4593-9321-df5a0840b48a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:53:55.536437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 876/1741] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:53:55 2022] DELETE /identity/v3/projects/ab11d7af982341ae894f3661f9f9f8f7 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:55.537783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07864e-f10e-4e40-8e6f-2fd0a4137e3e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:55.538217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07864e-f10e-4e40-8e6f-2fd0a4137e3e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:55.543505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07864e-f10e-4e40-8e6f-2fd0a4137e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:55.558841 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0e07864e-f10e-4e40-8e6f-2fd0a4137e3e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:55.560970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0e07864e-f10e-4e40-8e6f-2fd0a4137e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:55.642994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 866/1742] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:55.651539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e663-bb0b-4225-b4a4-2957fc0f173d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:55.652132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e663-bb0b-4225-b4a4-2957fc0f173d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:55.652505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6e663-bb0b-4225-b4a4-2957fc0f173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:55.677385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5cc6e663-bb0b-4225-b4a4-2957fc0f173d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:55.678529 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5cc6e663-bb0b-4225-b4a4-2957fc0f173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:55.688029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 877/1743] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:55.692222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07864e-f10e-4e40-8e6f-2fd0a4137e3e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:55.702150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07864e-f10e-4e40-8e6f-2fd0a4137e3e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:55.707138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:55.708014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:55.708304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:55.708583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:55.709390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:53:55.709702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:55.711336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:55.727110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 867/1744] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:53:55.729034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e663-bb0b-4225-b4a4-2957fc0f173d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:55.740101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6e663-bb0b-4225-b4a4-2957fc0f173d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:55.747216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb033-d5d2-44b5-b594-43ea8a0e365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:55.747867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb033-d5d2-44b5-b594-43ea8a0e365b admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:55.748154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb033-d5d2-44b5-b594-43ea8a0e365b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:55.748569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb033-d5d2-44b5-b594-43ea8a0e365b admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:55.749336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcb033-d5d2-44b5-b594-43ea8a0e365b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:55.751444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcb033-d5d2-44b5-b594-43ea8a0e365b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:55.868926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 878/1745] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:55 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:53:55.875960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:55.891217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55eeb2-ee26-4618-b50d-1e291a09eaf2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:55.902164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:55.950519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c23221-3a8e-44f2-9056-488be30f326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:55.951251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c23221-3a8e-44f2-9056-488be30f326c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:55.951566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c23221-3a8e-44f2-9056-488be30f326c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:55.951941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c23221-3a8e-44f2-9056-488be30f326c admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:55.952585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c23221-3a8e-44f2-9056-488be30f326c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:55.954481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c23221-3a8e-44f2-9056-488be30f326c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:56.074858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 868/1746] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:55 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:56.075075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb033-d5d2-44b5-b594-43ea8a0e365b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:56.086399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb033-d5d2-44b5-b594-43ea8a0e365b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:56.091977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702f794-60a8-43e2-a2b5-842e012de0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:56.092652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f794-60a8-43e2-a2b5-842e012de0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:56.092946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f794-60a8-43e2-a2b5-842e012de0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:56.093286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702f794-60a8-43e2-a2b5-842e012de0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:56.094016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702f794-60a8-43e2-a2b5-842e012de0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:56.095671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702f794-60a8-43e2-a2b5-842e012de0b7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:56.107529 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 879/1747] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:56.111515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c23221-3a8e-44f2-9056-488be30f326c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:56.121997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c23221-3a8e-44f2-9056-488be30f326c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:56.126898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a41904-21c4-4e0c-8959-1de1bfe1355b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:56.127652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41904-21c4-4e0c-8959-1de1bfe1355b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:56.127945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41904-21c4-4e0c-8959-1de1bfe1355b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:56.128230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41904-21c4-4e0c-8959-1de1bfe1355b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:56.129302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a41904-21c4-4e0c-8959-1de1bfe1355b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:56.132084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a41904-21c4-4e0c-8959-1de1bfe1355b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:56.147915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 869/1748] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:56.152236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702f794-60a8-43e2-a2b5-842e012de0b7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:56.165075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702f794-60a8-43e2-a2b5-842e012de0b7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:56.170423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649fe24-e39f-4520-9aa1-23b1057b3a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:56.171346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649fe24-e39f-4520-9aa1-23b1057b3a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:56.171725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649fe24-e39f-4520-9aa1-23b1057b3a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:56.171910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649fe24-e39f-4520-9aa1-23b1057b3a72 admin admin] PATH_INFO: `/v3/projects/3e0dfe7700844bfcadbc217a86e82410/users/89dd3592351e4eeab1f6d6005c25a7ba/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:56.187946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649fe24-e39f-4520-9aa1-23b1057b3a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0dfe7700844bfcadbc217a86e82410, user_id=89dd3592351e4eeab1f6d6005c25a7ba, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:56.190739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649fe24-e39f-4520-9aa1-23b1057b3a72 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:56.306152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 880/1749] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:56 2022] PUT /identity/v3/projects/3e0dfe7700844bfcadbc217a86e82410/users/89dd3592351e4eeab1f6d6005c25a7ba/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:53:56.316600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0508b-df03-4429-b1f1-ad158afcd46b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:56.316848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0508b-df03-4429-b1f1-ad158afcd46b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:56.317136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0508b-df03-4429-b1f1-ad158afcd46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:56.344944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-12f0508b-df03-4429-b1f1-ad158afcd46b None None] MFA Rules not processed for user `89dd3592351e4eeab1f6d6005c25a7ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:56.346153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-12f0508b-df03-4429-b1f1-ad158afcd46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:56.409130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 870/1750] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:56.413637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649fe24-e39f-4520-9aa1-23b1057b3a72 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:56.419352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:56.482497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649fe24-e39f-4520-9aa1-23b1057b3a72 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:56.487096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddfbe5-b198-4572-83ac-2814598a5b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:56.487790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddfbe5-b198-4572-83ac-2814598a5b58 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:56.490599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddfbe5-b198-4572-83ac-2814598a5b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:56.490599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddfbe5-b198-4572-83ac-2814598a5b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:56.490599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ddfbe5-b198-4572-83ac-2814598a5b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:56.491000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ddfbe5-b198-4572-83ac-2814598a5b58 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:56.594025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 881/1751] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:53:56 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:56.602759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0508b-df03-4429-b1f1-ad158afcd46b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:56.614256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0508b-df03-4429-b1f1-ad158afcd46b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:56.677276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10965610-67ab-4a50-83ef-76eba57c4766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:56.678015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10965610-67ab-4a50-83ef-76eba57c4766 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:56.678301 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10965610-67ab-4a50-83ef-76eba57c4766 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:56.678583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10965610-67ab-4a50-83ef-76eba57c4766 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:56.679273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10965610-67ab-4a50-83ef-76eba57c4766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:56.681232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10965610-67ab-4a50-83ef-76eba57c4766 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:56.779392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 871/1752] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:53:56 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:53:56.780644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddfbe5-b198-4572-83ac-2814598a5b58 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:56.791546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddfbe5-b198-4572-83ac-2814598a5b58 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:56.802871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b658a4-eced-4739-8c5c-3efbb589d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:56.803593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b658a4-eced-4739-8c5c-3efbb589d18e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:56.803878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b658a4-eced-4739-8c5c-3efbb589d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:56.804287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b658a4-eced-4739-8c5c-3efbb589d18e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:56.807339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b658a4-eced-4739-8c5c-3efbb589d18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:56.807339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b658a4-eced-4739-8c5c-3efbb589d18e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:56.818725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 882/1753] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:53:56.823035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10965610-67ab-4a50-83ef-76eba57c4766 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:56.839373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10965610-67ab-4a50-83ef-76eba57c4766 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:56.839373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd125a-f13b-4c83-ae0f-38a0602e3458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:56.839373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd125a-f13b-4c83-ae0f-38a0602e3458 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:56.839689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd125a-f13b-4c83-ae0f-38a0602e3458 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:56.839932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd125a-f13b-4c83-ae0f-38a0602e3458 admin admin] PATH_INFO: `/v3/projects/ee9d10b65bdb4f5f8d08a6f78ac20230/users/edd73086dce440bea1d4c45c928a087a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:56.859936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd125a-f13b-4c83-ae0f-38a0602e3458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9d10b65bdb4f5f8d08a6f78ac20230, user_id=edd73086dce440bea1d4c45c928a087a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:56.862733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd125a-f13b-4c83-ae0f-38a0602e3458 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:57.012123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 872/1754] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:56 2022] PUT /identity/v3/projects/ee9d10b65bdb4f5f8d08a6f78ac20230/users/edd73086dce440bea1d4c45c928a087a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:53:57.012123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:57.014175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b658a4-eced-4739-8c5c-3efbb589d18e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:57.045794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b658a4-eced-4739-8c5c-3efbb589d18e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:57.142353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6f16c-eb6d-4c1a-bdb5-5345e9f41575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:57.143991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f16c-eb6d-4c1a-bdb5-5345e9f41575 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:57.144508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f16c-eb6d-4c1a-bdb5-5345e9f41575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:57.146118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f16c-eb6d-4c1a-bdb5-5345e9f41575 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:57.147078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6f16c-eb6d-4c1a-bdb5-5345e9f41575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:57.148978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6f16c-eb6d-4c1a-bdb5-5345e9f41575 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:57.165442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 883/1755] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:53:57.167342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd125a-f13b-4c83-ae0f-38a0602e3458 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:57.178896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd125a-f13b-4c83-ae0f-38a0602e3458 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:57.185103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:57.186107 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:57.186393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:57.186670 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] PATH_INFO: `/v3/projects/ee9d10b65bdb4f5f8d08a6f78ac20230/users/edd73086dce440bea1d4c45c928a087a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:57.188696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9d10b65bdb4f5f8d08a6f78ac20230, user_id=edd73086dce440bea1d4c45c928a087a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:57.191567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:57.203355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 873/1756] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:53:57 2022] PUT /identity/v3/projects/ee9d10b65bdb4f5f8d08a6f78ac20230/users/edd73086dce440bea1d4c45c928a087a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:53:57.213858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a8daf-a47f-493b-b0c7-9c4b846c608a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:57.214175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a8daf-a47f-493b-b0c7-9c4b846c608a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:57.214456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a8daf-a47f-493b-b0c7-9c4b846c608a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:57.242184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3c0a8daf-a47f-493b-b0c7-9c4b846c608a None None] MFA Rules not processed for user `edd73086dce440bea1d4c45c928a087a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:53:57.243433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3c0a8daf-a47f-493b-b0c7-9c4b846c608a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:57.332953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 884/1757] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:53:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:53:57.346296 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:57.348011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:53:57.409902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:57.415771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b944abf-8227-4e8d-b20c-9e90ef1c57a8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:57.420100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c183d2-b074-4bf1-8c9e-d411c93cc96d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:57.420903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c183d2-b074-4bf1-8c9e-d411c93cc96d service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:57.421194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c183d2-b074-4bf1-8c9e-d411c93cc96d service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:57.421477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c183d2-b074-4bf1-8c9e-d411c93cc96d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:57.431189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c183d2-b074-4bf1-8c9e-d411c93cc96d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:57.432686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c183d2-b074-4bf1-8c9e-d411c93cc96d service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:57.444413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 874/1758] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:57.698895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a8daf-a47f-493b-b0c7-9c4b846c608a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:53:57.709433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a8daf-a47f-493b-b0c7-9c4b846c608a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:53:57.714748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967bb8c-38b0-4a3a-8f08-a41921224706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:53:57.715452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967bb8c-38b0-4a3a-8f08-a41921224706 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:53:57.715822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967bb8c-38b0-4a3a-8f08-a41921224706 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:53:57.716628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967bb8c-38b0-4a3a-8f08-a41921224706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:53:57.780781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967bb8c-38b0-4a3a-8f08-a41921224706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:53:57.782252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967bb8c-38b0-4a3a-8f08-a41921224706 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:53:57.793099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 885/1759] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:53:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:07.445548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=36,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:07.787680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=28 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:09.088174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ae31c-8710-41aa-b572-6ba77dffc5c4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:09.088956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ae31c-8710-41aa-b572-6ba77dffc5c4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:09.088956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ae31c-8710-41aa-b572-6ba77dffc5c4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:09.090530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 875/1760] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:54:09.105381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280a64-920f-4e31-9928-40e0f8c61221 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:09.105963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280a64-920f-4e31-9928-40e0f8c61221 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:09.106250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c280a64-920f-4e31-9928-40e0f8c61221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:09.125971 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4c280a64-920f-4e31-9928-40e0f8c61221 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:09.127217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4c280a64-920f-4e31-9928-40e0f8c61221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:09.212625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 886/1761] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:54:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:09.228134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ae31c-8710-41aa-b572-6ba77dffc5c4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:09.231174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-496ae31c-8710-41aa-b572-6ba77dffc5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:09.248761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ae31c-8710-41aa-b572-6ba77dffc5c4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:09.254215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6c17d-59a5-435c-8026-3cf1e5881adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:09.254904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6c17d-59a5-435c-8026-3cf1e5881adc service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:09.255193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6c17d-59a5-435c-8026-3cf1e5881adc service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:09.255504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6c17d-59a5-435c-8026-3cf1e5881adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:09.264965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6c17d-59a5-435c-8026-3cf1e5881adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:09.266562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6c17d-59a5-435c-8026-3cf1e5881adc service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:09.276803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 876/1762] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:11.448610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c280a64-920f-4e31-9928-40e0f8c61221 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:11.521484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c280a64-920f-4e31-9928-40e0f8c61221 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:11.526851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:11.527771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:11.528338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:11.528785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] PATH_INFO: `/v3/users/89dd3592351e4eeab1f6d6005c25a7ba` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:11.530151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89dd3592351e4eeab1f6d6005c25a7ba)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:11.533015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:11.762318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89dd3592351e4eeab1f6d6005c25a7ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:11.775979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89dd3592351e4eeab1f6d6005c25a7ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:11.776702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89dd3592351e4eeab1f6d6005c25a7ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:11.777317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89dd3592351e4eeab1f6d6005c25a7ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:11.784482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89dd3592351e4eeab1f6d6005c25a7ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:11.882484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 887/1763] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:11 2022] DELETE /identity/v3/users/89dd3592351e4eeab1f6d6005c25a7ba => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:11.890314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584d571-5e04-4435-925d-ba38d72dc407 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:11.890813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584d571-5e04-4435-925d-ba38d72dc407 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:11.890907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584d571-5e04-4435-925d-ba38d72dc407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:11.908575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6584d571-5e04-4435-925d-ba38d72dc407 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:11.909821 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6584d571-5e04-4435-925d-ba38d72dc407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:11.969311 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 877/1764] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:11.979596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:11.981445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:11.996983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:12.050754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0633f-3c04-49f1-b136-3e812e310d49 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:12.054418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0eea-1f21-46e8-ba6a-e913101a6847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:12.055069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0eea-1f21-46e8-ba6a-e913101a6847 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:12.055402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0eea-1f21-46e8-ba6a-e913101a6847 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:12.055681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0eea-1f21-46e8-ba6a-e913101a6847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:12.065110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e0eea-1f21-46e8-ba6a-e913101a6847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:12.066639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e0eea-1f21-46e8-ba6a-e913101a6847 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:12.077384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 888/1765] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:54:12.665880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584d571-5e04-4435-925d-ba38d72dc407 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:12.667283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6584d571-5e04-4435-925d-ba38d72dc407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:12.681631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584d571-5e04-4435-925d-ba38d72dc407 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:12.688288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:12.689263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:12.689553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:12.689828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] PATH_INFO: `/v3/projects/3e0dfe7700844bfcadbc217a86e82410` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:12.691231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0dfe7700844bfcadbc217a86e82410)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:12.694063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:12.833169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0dfe7700844bfcadbc217a86e82410', 'request_id': 'req-c9f4715c-460c-4423-bddd-4c8ac4821841'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:12.839505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0dfe7700844bfcadbc217a86e82410', 'request_id': 'req-c9f4715c-460c-4423-bddd-4c8ac4821841'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:12.845000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0dfe7700844bfcadbc217a86e82410', 'request_id': 'req-c9f4715c-460c-4423-bddd-4c8ac4821841'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:12.924825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] The token cache is being invalidate because project 3e0dfe7700844bfcadbc217a86e82410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:12.925438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:12.925752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:12.927369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 878/1766] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:12 2022] DELETE /identity/v3/projects/3e0dfe7700844bfcadbc217a86e82410 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:12.931510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0eea-1f21-46e8-ba6a-e913101a6847 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:12.941384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0eea-1f21-46e8-ba6a-e913101a6847 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:13.000887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:13.001797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:13.002132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:13.002411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] PATH_INFO: `/v3/users/edd73086dce440bea1d4c45c928a087a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:13.003930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd73086dce440bea1d4c45c928a087a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:13.005542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:13.200070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd73086dce440bea1d4c45c928a087a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:13.212590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd73086dce440bea1d4c45c928a087a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:13.213303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd73086dce440bea1d4c45c928a087a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:13.214159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd73086dce440bea1d4c45c928a087a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:13.221266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c826f0cc-ab06-480f-882f-233ed59a4401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd73086dce440bea1d4c45c928a087a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:13.296376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 889/1767] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:12 2022] DELETE /identity/v3/users/edd73086dce440bea1d4c45c928a087a => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:13.806279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:13.807813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:13.825524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4715c-460c-4423-bddd-4c8ac4821841 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:13.890245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:13.891329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:13.891703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:13.892086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] PATH_INFO: `/v3/projects/ee9d10b65bdb4f5f8d08a6f78ac20230` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:13.895598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9d10b65bdb4f5f8d08a6f78ac20230)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:13.897674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:13.912482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:14.038840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9d10b65bdb4f5f8d08a6f78ac20230', 'request_id': 'req-279394e9-80a3-4085-b924-151b29be3e47'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:14.045531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9d10b65bdb4f5f8d08a6f78ac20230', 'request_id': 'req-279394e9-80a3-4085-b924-151b29be3e47'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:14.051952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9d10b65bdb4f5f8d08a6f78ac20230', 'request_id': 'req-279394e9-80a3-4085-b924-151b29be3e47'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:14.169385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] The token cache is being invalidate because project ee9d10b65bdb4f5f8d08a6f78ac20230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:14.169786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:14.170532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-279394e9-80a3-4085-b924-151b29be3e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:14.172454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 879/1768] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:13 2022] DELETE /identity/v3/projects/ee9d10b65bdb4f5f8d08a6f78ac20230 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:14.177238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4f048-6ce8-42f3-b6c0-a174ca9faf48 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.177688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4f048-6ce8-42f3-b6c0-a174ca9faf48 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.177848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4f048-6ce8-42f3-b6c0-a174ca9faf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.196413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e2d4f048-6ce8-42f3-b6c0-a174ca9faf48 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:14.197674 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e2d4f048-6ce8-42f3-b6c0-a174ca9faf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:14.228626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:14.257795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 890/1769] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:14.266208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9d01e-08c3-4e89-be03-6ce4ab6f2d56 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.266498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9d01e-08c3-4e89-be03-6ce4ab6f2d56 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.266775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9d01e-08c3-4e89-be03-6ce4ab6f2d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.285641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ccc9d01e-08c3-4e89-be03-6ce4ab6f2d56 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:14.286845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ccc9d01e-08c3-4e89-be03-6ce4ab6f2d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:14.295133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 880/1770] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:14.299162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4f048-6ce8-42f3-b6c0-a174ca9faf48 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:14.309153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4f048-6ce8-42f3-b6c0-a174ca9faf48 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:14.313920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:14.322433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.322433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.322433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.322433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:14.322433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:14.322433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:14.329784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 891/1771] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:54:14.334036 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9d01e-08c3-4e89-be03-6ce4ab6f2d56 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:14.344759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9d01e-08c3-4e89-be03-6ce4ab6f2d56 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:14.349333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1588566-73cf-44d7-8960-ac6142c68294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:14.349982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588566-73cf-44d7-8960-ac6142c68294 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.350264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588566-73cf-44d7-8960-ac6142c68294 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.350540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588566-73cf-44d7-8960-ac6142c68294 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.351218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1588566-73cf-44d7-8960-ac6142c68294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:14.353143 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1588566-73cf-44d7-8960-ac6142c68294 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:14.434663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 881/1772] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:14 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:14.439054 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:14.448922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a39729-e233-4f59-87f4-c427f6df20ad admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:14.511646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7867-c89f-4f4b-b3ad-41afe634cc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:14.512577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7867-c89f-4f4b-b3ad-41afe634cc6e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.512933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7867-c89f-4f4b-b3ad-41afe634cc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.513283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7867-c89f-4f4b-b3ad-41afe634cc6e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.514858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d7867-c89f-4f4b-b3ad-41afe634cc6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:14.516915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d7867-c89f-4f4b-b3ad-41afe634cc6e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:14.598884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 892/1773] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:14 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:14.603216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1588566-73cf-44d7-8960-ac6142c68294 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:14.614130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1588566-73cf-44d7-8960-ac6142c68294 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:14.619541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbe895-a5d4-49cb-9c2a-a47817262ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:14.620214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe895-a5d4-49cb-9c2a-a47817262ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.620493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe895-a5d4-49cb-9c2a-a47817262ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.620769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe895-a5d4-49cb-9c2a-a47817262ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.621516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbe895-a5d4-49cb-9c2a-a47817262ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:14.623724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbe895-a5d4-49cb-9c2a-a47817262ce1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:14.635067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 882/1774] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:14.639723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7867-c89f-4f4b-b3ad-41afe634cc6e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:14.650496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d7867-c89f-4f4b-b3ad-41afe634cc6e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:14.655253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21446-80e4-4b4d-9a3f-5217e6946b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:14.656040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21446-80e4-4b4d-9a3f-5217e6946b36 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.656332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21446-80e4-4b4d-9a3f-5217e6946b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.656611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21446-80e4-4b4d-9a3f-5217e6946b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.657334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a21446-80e4-4b4d-9a3f-5217e6946b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:14.658932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a21446-80e4-4b4d-9a3f-5217e6946b36 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:14.669199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 893/1775] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:54:14.673241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbe895-a5d4-49cb-9c2a-a47817262ce1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:14.683119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbe895-a5d4-49cb-9c2a-a47817262ce1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:14.688373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1ae92-81a9-4e98-9eb8-a64c5d4ebfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:14.689254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1ae92-81a9-4e98-9eb8-a64c5d4ebfcd admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.689546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1ae92-81a9-4e98-9eb8-a64c5d4ebfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.689991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1ae92-81a9-4e98-9eb8-a64c5d4ebfcd admin admin] PATH_INFO: `/v3/projects/351f7931e3e14d248f31842f245680ea/users/557bdb19a350449988b49dd298451f18/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.705005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1ae92-81a9-4e98-9eb8-a64c5d4ebfcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=351f7931e3e14d248f31842f245680ea, user_id=557bdb19a350449988b49dd298451f18, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:14.707740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1ae92-81a9-4e98-9eb8-a64c5d4ebfcd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:14.804361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 883/1776] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:14 2022] PUT /identity/v3/projects/351f7931e3e14d248f31842f245680ea/users/557bdb19a350449988b49dd298451f18/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:14.814790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db6640-b3f8-491b-8026-4990c92dceea None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.815089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db6640-b3f8-491b-8026-4990c92dceea None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.815420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db6640-b3f8-491b-8026-4990c92dceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.850765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a8db6640-b3f8-491b-8026-4990c92dceea None None] MFA Rules not processed for user `557bdb19a350449988b49dd298451f18`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:14.850765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a8db6640-b3f8-491b-8026-4990c92dceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:14.919020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 894/1777] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:14.923676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1ae92-81a9-4e98-9eb8-a64c5d4ebfcd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:14.987056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1ae92-81a9-4e98-9eb8-a64c5d4ebfcd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:14.991730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b5c50-0d74-4c76-bf9f-994f422178ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:14.992390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b5c50-0d74-4c76-bf9f-994f422178ac admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:14.992674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b5c50-0d74-4c76-bf9f-994f422178ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:14.992951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b5c50-0d74-4c76-bf9f-994f422178ac admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:14.993630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b5c50-0d74-4c76-bf9f-994f422178ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:14.995571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b5c50-0d74-4c76-bf9f-994f422178ac admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:15.073776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 884/1778] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:14 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:15.078439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db6640-b3f8-491b-8026-4990c92dceea None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:15.091229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db6640-b3f8-491b-8026-4990c92dceea None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:15.128825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:15.158522 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985c517-178b-4dde-9212-787c0b8dd919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:15.159208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985c517-178b-4dde-9212-787c0b8dd919 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:15.159540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985c517-178b-4dde-9212-787c0b8dd919 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:15.159807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985c517-178b-4dde-9212-787c0b8dd919 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:15.160494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985c517-178b-4dde-9212-787c0b8dd919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:15.162413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985c517-178b-4dde-9212-787c0b8dd919 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:15.265346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 895/1779] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:15 2022] POST /identity/v3/users => generated 296 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:15.269493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b5c50-0d74-4c76-bf9f-994f422178ac admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:15.280868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b5c50-0d74-4c76-bf9f-994f422178ac admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:15.286217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96d79-cb30-4237-9656-d0a7d1108908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:15.287007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96d79-cb30-4237-9656-d0a7d1108908 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:15.287410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96d79-cb30-4237-9656-d0a7d1108908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:15.287668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96d79-cb30-4237-9656-d0a7d1108908 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:15.288386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a96d79-cb30-4237-9656-d0a7d1108908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:15.289992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a96d79-cb30-4237-9656-d0a7d1108908 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:15.304043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 885/1780] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:15.306922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985c517-178b-4dde-9212-787c0b8dd919 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:15.317045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985c517-178b-4dde-9212-787c0b8dd919 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:15.321833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4faa08-a3df-4831-a4d3-af36f9e6cf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:15.322724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4faa08-a3df-4831-a4d3-af36f9e6cf4b admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:15.323007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4faa08-a3df-4831-a4d3-af36f9e6cf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:15.323319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4faa08-a3df-4831-a4d3-af36f9e6cf4b admin admin] PATH_INFO: `/v3/projects/a28e9205d6e548158a3d35de6166297d/users/e8ad4acdc9b34c9085c680835859284a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:15.343671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4faa08-a3df-4831-a4d3-af36f9e6cf4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28e9205d6e548158a3d35de6166297d, user_id=e8ad4acdc9b34c9085c680835859284a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:15.346632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4faa08-a3df-4831-a4d3-af36f9e6cf4b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:15.417074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 896/1781] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:15 2022] PUT /identity/v3/projects/a28e9205d6e548158a3d35de6166297d/users/e8ad4acdc9b34c9085c680835859284a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:15.421318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96d79-cb30-4237-9656-d0a7d1108908 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:15.431873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96d79-cb30-4237-9656-d0a7d1108908 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:15.440522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:15.491117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdc84c-3875-431a-ae1a-1534a67cabe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:15.491825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdc84c-3875-431a-ae1a-1534a67cabe3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:15.492114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdc84c-3875-431a-ae1a-1534a67cabe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:15.492397 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdc84c-3875-431a-ae1a-1534a67cabe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:15.493175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdc84c-3875-431a-ae1a-1534a67cabe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:15.494842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdc84c-3875-431a-ae1a-1534a67cabe3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:15.506355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 886/1782] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:15.510695 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4faa08-a3df-4831-a4d3-af36f9e6cf4b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:15.528472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4faa08-a3df-4831-a4d3-af36f9e6cf4b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:15.534633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:15.539143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:15.539143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:15.539143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] PATH_INFO: `/v3/projects/a28e9205d6e548158a3d35de6166297d/users/e8ad4acdc9b34c9085c680835859284a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:15.539143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28e9205d6e548158a3d35de6166297d, user_id=e8ad4acdc9b34c9085c680835859284a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:15.541251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:15.556317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 897/1783] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:15 2022] PUT /identity/v3/projects/a28e9205d6e548158a3d35de6166297d/users/e8ad4acdc9b34c9085c680835859284a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:15.569141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf0b4e-c79e-4924-a6d5-aab134f11641 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:15.569441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf0b4e-c79e-4924-a6d5-aab134f11641 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:15.569723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf0b4e-c79e-4924-a6d5-aab134f11641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:15.601233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f4bf0b4e-c79e-4924-a6d5-aab134f11641 None None] MFA Rules not processed for user `e8ad4acdc9b34c9085c680835859284a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:15.602477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f4bf0b4e-c79e-4924-a6d5-aab134f11641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:15.672434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 887/1784] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:15.683544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:15.685352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:15.743929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73446f6-bce0-4c2e-8883-9b296cc401d6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:15.747610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15759faf-4617-4006-8117-7787167d69b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:15.748834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759faf-4617-4006-8117-7787167d69b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:15.749198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759faf-4617-4006-8117-7787167d69b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:15.749401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15759faf-4617-4006-8117-7787167d69b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:15.758555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15759faf-4617-4006-8117-7787167d69b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:15.760030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15759faf-4617-4006-8117-7787167d69b0 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:15.769802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 898/1785] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:16.327849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf0b4e-c79e-4924-a6d5-aab134f11641 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:16.395437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf0b4e-c79e-4924-a6d5-aab134f11641 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:16.400182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:16.401059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:16.401349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:16.401629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] PATH_INFO: `/v3/users/557bdb19a350449988b49dd298451f18` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:16.408004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=557bdb19a350449988b49dd298451f18)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:16.408004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:16.659335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:16.669396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '557bdb19a350449988b49dd298451f18'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:16.681493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '557bdb19a350449988b49dd298451f18'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:16.682192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '557bdb19a350449988b49dd298451f18'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:16.682861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '557bdb19a350449988b49dd298451f18'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:16.690204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '557bdb19a350449988b49dd298451f18'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:16.765204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 888/1786] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:16 2022] DELETE /identity/v3/users/557bdb19a350449988b49dd298451f18 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:16.770284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e12c12-b2bb-4895-bfdd-f1f20f96d710 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:16.770766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e12c12-b2bb-4895-bfdd-f1f20f96d710 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:16.770856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e12c12-b2bb-4895-bfdd-f1f20f96d710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:16.788710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-36e12c12-b2bb-4895-bfdd-f1f20f96d710 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:16.790031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-36e12c12-b2bb-4895-bfdd-f1f20f96d710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:16.850983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 899/1787] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:16.861404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:16.862879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:16.933902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a7afd-2bf2-494a-ba7d-c30fff4c0c66 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:16.938200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:16.938850 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:16.939139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:16.939456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:16.948940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:16.950395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:16.960888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 889/1788] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:17.505779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e12c12-b2bb-4895-bfdd-f1f20f96d710 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:17.507427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-36e12c12-b2bb-4895-bfdd-f1f20f96d710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:17.523108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e12c12-b2bb-4895-bfdd-f1f20f96d710 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:17.534489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:17.534489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:17.534489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:17.534489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] PATH_INFO: `/v3/projects/351f7931e3e14d248f31842f245680ea` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:17.534896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=351f7931e3e14d248f31842f245680ea)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:17.539489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:17.571395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:17.699591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:17.704574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:17.719077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8496cbd-9e6d-4770-8b45-ab046498189f service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:17.720435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '351f7931e3e14d248f31842f245680ea', 'request_id': 'req-ee3cc3ca-53d4-461a-a10a-328ac33db445'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:17.726517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '351f7931e3e14d248f31842f245680ea', 'request_id': 'req-ee3cc3ca-53d4-461a-a10a-328ac33db445'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:17.776704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:17.777576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:17.777890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:17.778189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] PATH_INFO: `/v3/users/7b4b2714c6cf4d3885843766d24d3485` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:17.779446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4b2714c6cf4d3885843766d24d3485)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:17.781441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:17.825227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '351f7931e3e14d248f31842f245680ea', 'request_id': 'req-ee3cc3ca-53d4-461a-a10a-328ac33db445'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:17.837617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] The token cache is being invalidate because project 351f7931e3e14d248f31842f245680ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:17.837909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:17.838557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:17.840399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 900/1789] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:17 2022] DELETE /identity/v3/projects/351f7931e3e14d248f31842f245680ea => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:17.844205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:17.845598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:17.858706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cc3ca-53d4-461a-a10a-328ac33db445 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:17.863445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:17.864411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:17.864771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:17.865124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] PATH_INFO: `/v3/users/e8ad4acdc9b34c9085c680835859284a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:17.866403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ad4acdc9b34c9085c680835859284a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:17.868435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:18.060641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4b2714c6cf4d3885843766d24d3485'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.078120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:18.084682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4b2714c6cf4d3885843766d24d3485'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.085509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4b2714c6cf4d3885843766d24d3485'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.086229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4b2714c6cf4d3885843766d24d3485'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.099073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a4893fbc-6b69-4257-a2ba-db2467e9a1d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4b2714c6cf4d3885843766d24d3485'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.164457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ad4acdc9b34c9085c680835859284a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.188997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ad4acdc9b34c9085c680835859284a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.190115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ad4acdc9b34c9085c680835859284a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.190718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ad4acdc9b34c9085c680835859284a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.205333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 890/1790] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:17 2022] DELETE /identity/v3/users/7b4b2714c6cf4d3885843766d24d3485 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:18.205981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ad4acdc9b34c9085c680835859284a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:18.209802 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f66f-5d0a-4e0d-9188-4c889f35ae27 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:18.210231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f66f-5d0a-4e0d-9188-4c889f35ae27 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:18.210527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f66f-5d0a-4e0d-9188-4c889f35ae27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:18.229941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3c13f66f-5d0a-4e0d-9188-4c889f35ae27 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:18.231161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3c13f66f-5d0a-4e0d-9188-4c889f35ae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:18.295764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 891/1791] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:54:18.295916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 901/1792] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:17 2022] DELETE /identity/v3/users/e8ad4acdc9b34c9085c680835859284a => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:18.304797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13f66f-5d0a-4e0d-9188-4c889f35ae27 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:18.307430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3c13f66f-5d0a-4e0d-9188-4c889f35ae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:18.367951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13f66f-5d0a-4e0d-9188-4c889f35ae27 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:18.422976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85a75c-1765-42cd-9c7b-957c74e672c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:18.430698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85a75c-1765-42cd-9c7b-957c74e672c0 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:18.430698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85a75c-1765-42cd-9c7b-957c74e672c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:18.430698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85a75c-1765-42cd-9c7b-957c74e672c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:18.468113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:18.484466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85a75c-1765-42cd-9c7b-957c74e672c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:18.485925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85a75c-1765-42cd-9c7b-957c74e672c0 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:18.496178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 892/1793] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:18.951364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:18.952837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:18.972847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d1aa-d145-48fa-81c0-e0b0a3859b2f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:18.984829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:18.986150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:18.986762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:18.987239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] PATH_INFO: `/v3/projects/a28e9205d6e548158a3d35de6166297d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:18.988942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28e9205d6e548158a3d35de6166297d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:18.991256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.160162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28e9205d6e548158a3d35de6166297d', 'request_id': 'req-f1485092-9ebf-4db0-9a30-234df1572934'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.165617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28e9205d6e548158a3d35de6166297d', 'request_id': 'req-f1485092-9ebf-4db0-9a30-234df1572934'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.289992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28e9205d6e548158a3d35de6166297d', 'request_id': 'req-f1485092-9ebf-4db0-9a30-234df1572934'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.302629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] The token cache is being invalidate because project a28e9205d6e548158a3d35de6166297d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:19.302920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.303629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.305244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 902/1794] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:18 2022] DELETE /identity/v3/projects/a28e9205d6e548158a3d35de6166297d => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:19.311532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532a45-26a7-4418-a1a8-326048917c47 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.311985 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532a45-26a7-4418-a1a8-326048917c47 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.312075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5532a45-26a7-4418-a1a8-326048917c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.328122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.330240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:19.337524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=73,keystone:DELETE=24,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:19.342667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f5532a45-26a7-4418-a1a8-326048917c47 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:19.343146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f5532a45-26a7-4418-a1a8-326048917c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:19.438378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1485092-9ebf-4db0-9a30-234df1572934 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.443764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 893/1795] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:19.443899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.444115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.444399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.444678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] PATH_INFO: `/v3/projects/61d33542b62342ddb514e4b35c6e94b0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.446534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d33542b62342ddb514e4b35c6e94b0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.447861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f554b8-61f8-4bb0-b44b-262a4c886c5f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.447973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.448197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f554b8-61f8-4bb0-b44b-262a4c886c5f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.448630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f554b8-61f8-4bb0-b44b-262a4c886c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.468860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e7f554b8-61f8-4bb0-b44b-262a4c886c5f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:19.470338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e7f554b8-61f8-4bb0-b44b-262a4c886c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:19.478575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 894/1796] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:54:19.482733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f554b8-61f8-4bb0-b44b-262a4c886c5f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.491647 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f554b8-61f8-4bb0-b44b-262a4c886c5f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.498163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.498818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.499106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.499418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.500177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:19.500798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.502088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.519203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 895/1797] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:54:19.520646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.531192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed6fed-2d8d-4a5e-bd56-10ca3e14807e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.536608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47aa78b-ada6-4480-8b5d-08eb15a19870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.537685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47aa78b-ada6-4480-8b5d-08eb15a19870 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.538190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47aa78b-ada6-4480-8b5d-08eb15a19870 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.538732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47aa78b-ada6-4480-8b5d-08eb15a19870 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.540146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47aa78b-ada6-4480-8b5d-08eb15a19870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.543882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47aa78b-ada6-4480-8b5d-08eb15a19870 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.605373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d33542b62342ddb514e4b35c6e94b0', 'request_id': 'req-6a5098a0-c585-4344-ab8e-62154cd5fa01'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.612338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d33542b62342ddb514e4b35c6e94b0', 'request_id': 'req-6a5098a0-c585-4344-ab8e-62154cd5fa01'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.683428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 896/1798] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:19 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:54:19.684403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47aa78b-ada6-4480-8b5d-08eb15a19870 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.702126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d33542b62342ddb514e4b35c6e94b0', 'request_id': 'req-6a5098a0-c585-4344-ab8e-62154cd5fa01'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.717497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] The token cache is being invalidate because project 61d33542b62342ddb514e4b35c6e94b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:19.717878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.718630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6a5098a0-c585-4344-ab8e-62154cd5fa01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:19.720734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 903/1799] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:19 2022] DELETE /identity/v3/projects/61d33542b62342ddb514e4b35c6e94b0 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:19.726688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c470f31-a152-4999-baeb-b28f54e3e7c2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.726688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c470f31-a152-4999-baeb-b28f54e3e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.726688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c470f31-a152-4999-baeb-b28f54e3e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.747401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0c470f31-a152-4999-baeb-b28f54e3e7c2 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:19.749146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0c470f31-a152-4999-baeb-b28f54e3e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:19.757599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47aa78b-ada6-4480-8b5d-08eb15a19870 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.761403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 904/1800] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:19.762873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.765564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.765910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.768067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.768067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.774480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.774589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18fb51-15ee-40f7-9f65-e3a03968b063 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.774589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18fb51-15ee-40f7-9f65-e3a03968b063 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.774589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18fb51-15ee-40f7-9f65-e3a03968b063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.798416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4c18fb51-15ee-40f7-9f65-e3a03968b063 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:19.798416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4c18fb51-15ee-40f7-9f65-e3a03968b063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:19.807508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 905/1801] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:19.812092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18fb51-15ee-40f7-9f65-e3a03968b063 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.821990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18fb51-15ee-40f7-9f65-e3a03968b063 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.827860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.828539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.829349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.829806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.830825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:19.831141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.832840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.847031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 906/1802] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:54:19.851162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.862758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662a6f6-7cc9-47b8-a001-e63f171512a0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.868154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ad1eb-2af2-4ef0-868d-3a005e506782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.869823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad1eb-2af2-4ef0-868d-3a005e506782 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.869823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad1eb-2af2-4ef0-868d-3a005e506782 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.870119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ad1eb-2af2-4ef0-868d-3a005e506782 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.874280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 897/1803] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:19 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:54:19.874432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ad1eb-2af2-4ef0-868d-3a005e506782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.874432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ad1eb-2af2-4ef0-868d-3a005e506782 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.875642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.877323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:19.892619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c850bca-3b88-4213-9525-6899d786a66d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.897993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6a5df-8778-4a05-af33-9c77280687c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.898677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6a5df-8778-4a05-af33-9c77280687c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.898970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6a5df-8778-4a05-af33-9c77280687c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.900589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6a5df-8778-4a05-af33-9c77280687c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.900589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6a5df-8778-4a05-af33-9c77280687c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.901640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f6a5df-8778-4a05-af33-9c77280687c3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.912986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 898/1804] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:19.917038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6a5df-8778-4a05-af33-9c77280687c3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.927684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6a5df-8778-4a05-af33-9c77280687c3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.932998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80448671-d6fb-43c6-a4fd-f067ec7b4905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.933663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80448671-d6fb-43c6-a4fd-f067ec7b4905 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.933954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80448671-d6fb-43c6-a4fd-f067ec7b4905 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.934237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80448671-d6fb-43c6-a4fd-f067ec7b4905 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.934966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80448671-d6fb-43c6-a4fd-f067ec7b4905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.937323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80448671-d6fb-43c6-a4fd-f067ec7b4905 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.952010 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 899/1805] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:54:19.955752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80448671-d6fb-43c6-a4fd-f067ec7b4905 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:19.965326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80448671-d6fb-43c6-a4fd-f067ec7b4905 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:19.972647 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e631e2-0bc3-4135-8367-54965b475686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:19.973655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e631e2-0bc3-4135-8367-54965b475686 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:19.973885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e631e2-0bc3-4135-8367-54965b475686 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:19.974170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e631e2-0bc3-4135-8367-54965b475686 admin admin] PATH_INFO: `/v3/projects/6df5a6ce22e54ad7ae77ea8ca1b03bd7/users/50e4f561035e4f739c322dd8dbf745bc/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:19.976842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:19.994279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 907/1806] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:19 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:54:19.994806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e631e2-0bc3-4135-8367-54965b475686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df5a6ce22e54ad7ae77ea8ca1b03bd7, user_id=50e4f561035e4f739c322dd8dbf745bc, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:19.997667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e631e2-0bc3-4135-8367-54965b475686 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:19.999103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ad1eb-2af2-4ef0-868d-3a005e506782 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:20.061855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ad1eb-2af2-4ef0-868d-3a005e506782 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:20.067352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648f72d-1801-478e-9428-d78f268c86c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:20.068049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648f72d-1801-478e-9428-d78f268c86c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:20.068469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648f72d-1801-478e-9428-d78f268c86c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:20.068753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648f72d-1801-478e-9428-d78f268c86c1 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:20.069502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648f72d-1801-478e-9428-d78f268c86c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:20.071562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 900/1807] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:19 2022] PUT /identity/v3/projects/6df5a6ce22e54ad7ae77ea8ca1b03bd7/users/50e4f561035e4f739c322dd8dbf745bc/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:20.071787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648f72d-1801-478e-9428-d78f268c86c1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:20.082208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba71d0-3166-4e74-9d8d-78fbeccc0770 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:20.082584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba71d0-3166-4e74-9d8d-78fbeccc0770 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:20.082984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba71d0-3166-4e74-9d8d-78fbeccc0770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:20.109948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2dba71d0-3166-4e74-9d8d-78fbeccc0770 None None] MFA Rules not processed for user `50e4f561035e4f739c322dd8dbf745bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:20.111158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2dba71d0-3166-4e74-9d8d-78fbeccc0770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:20.176835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 901/1808] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:54:20.188950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba71d0-3166-4e74-9d8d-78fbeccc0770 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:20.190694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2dba71d0-3166-4e74-9d8d-78fbeccc0770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:20.193304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 908/1809] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:19 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:20.197820 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648f72d-1801-478e-9428-d78f268c86c1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:20.208656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648f72d-1801-478e-9428-d78f268c86c1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:20.258778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba71d0-3166-4e74-9d8d-78fbeccc0770 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:20.263693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682aa8-0b75-465c-b02e-488b5b8a3766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:20.264351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682aa8-0b75-465c-b02e-488b5b8a3766 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:20.264643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682aa8-0b75-465c-b02e-488b5b8a3766 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:20.264923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04682aa8-0b75-465c-b02e-488b5b8a3766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:20.272970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a6486-262b-4ec5-9856-8fc9c410aed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:20.273659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6486-262b-4ec5-9856-8fc9c410aed2 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:20.273953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6486-262b-4ec5-9856-8fc9c410aed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:20.274238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6486-262b-4ec5-9856-8fc9c410aed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:20.274828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04682aa8-0b75-465c-b02e-488b5b8a3766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:20.274991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a6486-262b-4ec5-9856-8fc9c410aed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:20.276602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04682aa8-0b75-465c-b02e-488b5b8a3766 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:20.276707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a6486-262b-4ec5-9856-8fc9c410aed2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:20.287629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 902/1810] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:20.287756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 909/1811] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:54:20.291371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682aa8-0b75-465c-b02e-488b5b8a3766 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:20.302719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04682aa8-0b75-465c-b02e-488b5b8a3766 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:20.307417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681696b2-5c79-48d3-8193-0b2e7949d587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:20.308072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681696b2-5c79-48d3-8193-0b2e7949d587 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:20.308359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681696b2-5c79-48d3-8193-0b2e7949d587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:20.308642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681696b2-5c79-48d3-8193-0b2e7949d587 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:20.309361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681696b2-5c79-48d3-8193-0b2e7949d587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:20.311029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681696b2-5c79-48d3-8193-0b2e7949d587 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:20.321637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 903/1812] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:54:20.325462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a6486-262b-4ec5-9856-8fc9c410aed2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:20.335986 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a6486-262b-4ec5-9856-8fc9c410aed2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:20.341054 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:20.342583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:20.342873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:20.343157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] PATH_INFO: `/v3/projects/650e9a1656a0417e844fb6c13b3fc9cf/users/c132cda101514dde9f5dc2f8c7e70ce2/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:20.361788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=650e9a1656a0417e844fb6c13b3fc9cf, user_id=c132cda101514dde9f5dc2f8c7e70ce2, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:20.364539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:20.462899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 910/1813] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:20 2022] PUT /identity/v3/projects/650e9a1656a0417e844fb6c13b3fc9cf/users/c132cda101514dde9f5dc2f8c7e70ce2/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:20.473805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acced1-67b5-4c04-b9f1-fea8f05b1284 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:20.474100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acced1-67b5-4c04-b9f1-fea8f05b1284 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:20.474381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acced1-67b5-4c04-b9f1-fea8f05b1284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:20.501617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b1acced1-67b5-4c04-b9f1-fea8f05b1284 None None] MFA Rules not processed for user `c132cda101514dde9f5dc2f8c7e70ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:20.502862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b1acced1-67b5-4c04-b9f1-fea8f05b1284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:20.568467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 904/1814] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:20.580088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:20.581959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:20.656537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2be648-fdb8-40f0-b7a0-f6b114627605 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:20.663045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b9a40-cd18-475f-aa48-d6db9da7e2f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:20.663747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9a40-cd18-475f-aa48-d6db9da7e2f5 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:20.664038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9a40-cd18-475f-aa48-d6db9da7e2f5 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:20.664322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b9a40-cd18-475f-aa48-d6db9da7e2f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:20.673378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b9a40-cd18-475f-aa48-d6db9da7e2f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:20.674989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b9a40-cd18-475f-aa48-d6db9da7e2f5 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:20.685122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 911/1815] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:20.811694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:20.870039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:20.922993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acced1-67b5-4c04-b9f1-fea8f05b1284 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:21.002689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acced1-67b5-4c04-b9f1-fea8f05b1284 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:21.028049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:21.029278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:21.030141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:21.030496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] PATH_INFO: `/v3/users/c132cda101514dde9f5dc2f8c7e70ce2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:21.032222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c132cda101514dde9f5dc2f8c7e70ce2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:21.034344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:21.273507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c132cda101514dde9f5dc2f8c7e70ce2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:21.286329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c132cda101514dde9f5dc2f8c7e70ce2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:21.287067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c132cda101514dde9f5dc2f8c7e70ce2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:21.287715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c132cda101514dde9f5dc2f8c7e70ce2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:21.295776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c132cda101514dde9f5dc2f8c7e70ce2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:21.439523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 905/1816] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:20 2022] DELETE /identity/v3/users/c132cda101514dde9f5dc2f8c7e70ce2 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:21.444023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf15137-7977-4019-b38c-7df6bb2b0b3a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:21.444445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf15137-7977-4019-b38c-7df6bb2b0b3a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:21.450883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf15137-7977-4019-b38c-7df6bb2b0b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:21.462590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0bf15137-7977-4019-b38c-7df6bb2b0b3a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:21.463869 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0bf15137-7977-4019-b38c-7df6bb2b0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:21.522893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 912/1817] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:21.533421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:21.534998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:21.594944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7bbfe-06b0-4966-a7e5-7db013975724 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:21.598715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e825842-ab52-49fd-8865-e7a2e79d2a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:21.599416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e825842-ab52-49fd-8865-e7a2e79d2a5f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:21.599705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e825842-ab52-49fd-8865-e7a2e79d2a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:21.599979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e825842-ab52-49fd-8865-e7a2e79d2a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:21.609663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e825842-ab52-49fd-8865-e7a2e79d2a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:21.611127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e825842-ab52-49fd-8865-e7a2e79d2a5f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:21.620944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 906/1818] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:21.826502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:22.183755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf15137-7977-4019-b38c-7df6bb2b0b3a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:22.185195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0bf15137-7977-4019-b38c-7df6bb2b0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:22.197772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf15137-7977-4019-b38c-7df6bb2b0b3a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:22.203880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:22.204823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:22.205177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:22.205523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] PATH_INFO: `/v3/projects/650e9a1656a0417e844fb6c13b3fc9cf` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:22.206976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=650e9a1656a0417e844fb6c13b3fc9cf)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:22.209157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:22.359539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '650e9a1656a0417e844fb6c13b3fc9cf', 'request_id': 'req-83cda195-9efa-4ead-8816-0a66a5571ec4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:22.366660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '650e9a1656a0417e844fb6c13b3fc9cf', 'request_id': 'req-83cda195-9efa-4ead-8816-0a66a5571ec4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:22.456384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '650e9a1656a0417e844fb6c13b3fc9cf', 'request_id': 'req-83cda195-9efa-4ead-8816-0a66a5571ec4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:22.469678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] The token cache is being invalidate because project 650e9a1656a0417e844fb6c13b3fc9cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:22.470034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:22.470758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83cda195-9efa-4ead-8816-0a66a5571ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:22.472445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 913/1819] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:22 2022] DELETE /identity/v3/projects/650e9a1656a0417e844fb6c13b3fc9cf => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:22.477317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b374873-f772-42e4-b8ec-240ac53b4069 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:22.477808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b374873-f772-42e4-b8ec-240ac53b4069 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:22.477917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b374873-f772-42e4-b8ec-240ac53b4069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:22.495581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7b374873-f772-42e4-b8ec-240ac53b4069 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:22.496776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7b374873-f772-42e4-b8ec-240ac53b4069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:22.507349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:22.557435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 907/1820] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:22.566904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2c465-9e40-4c53-82de-3bc0a8f916df None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:22.567191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2c465-9e40-4c53-82de-3bc0a8f916df None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:22.567502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2c465-9e40-4c53-82de-3bc0a8f916df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:22.586189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ddd2c465-9e40-4c53-82de-3bc0a8f916df None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:22.587691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ddd2c465-9e40-4c53-82de-3bc0a8f916df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:22.595986 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 914/1821] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:22.600339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b374873-f772-42e4-b8ec-240ac53b4069 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:22.610510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b374873-f772-42e4-b8ec-240ac53b4069 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:22.615420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:22.620414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:22.620414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:22.620414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:22.620414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:22.620414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:22.620414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:22.631956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 908/1822] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:54:22.636255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2c465-9e40-4c53-82de-3bc0a8f916df None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:22.645638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2c465-9e40-4c53-82de-3bc0a8f916df None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:22.650322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c799d-2072-4bc4-bfc1-ed790148fb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:22.650973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c799d-2072-4bc4-bfc1-ed790148fb1f admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:22.651257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c799d-2072-4bc4-bfc1-ed790148fb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:22.651590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c799d-2072-4bc4-bfc1-ed790148fb1f admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:22.652284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c799d-2072-4bc4-bfc1-ed790148fb1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:22.660547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c799d-2072-4bc4-bfc1-ed790148fb1f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:22.783350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 915/1823] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:22 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:22.787437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:22.797982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820890c-cb75-472d-bd7b-2310adb3221c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:22.858321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb939322-83ca-4ebd-be58-a47a09b8291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:22.859009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb939322-83ca-4ebd-be58-a47a09b8291e admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:22.859321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb939322-83ca-4ebd-be58-a47a09b8291e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:22.859624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb939322-83ca-4ebd-be58-a47a09b8291e admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:22.860323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb939322-83ca-4ebd-be58-a47a09b8291e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:22.862274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb939322-83ca-4ebd-be58-a47a09b8291e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:22.982166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 909/1824] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:22 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:54:22.995565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c799d-2072-4bc4-bfc1-ed790148fb1f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:23.006730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c799d-2072-4bc4-bfc1-ed790148fb1f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:23.011256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dfd0b-bef7-4f11-a6a1-dd9697e13dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:23.011934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dfd0b-bef7-4f11-a6a1-dd9697e13dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:23.012215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dfd0b-bef7-4f11-a6a1-dd9697e13dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:23.012488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dfd0b-bef7-4f11-a6a1-dd9697e13dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:23.013200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dfd0b-bef7-4f11-a6a1-dd9697e13dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:23.015760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dfd0b-bef7-4f11-a6a1-dd9697e13dd4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:23.027075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 916/1825] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:23.031430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb939322-83ca-4ebd-be58-a47a09b8291e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:23.041682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb939322-83ca-4ebd-be58-a47a09b8291e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:23.046996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b8d4-8224-4dc8-8d67-4a1b649d8598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:23.047705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b8d4-8224-4dc8-8d67-4a1b649d8598 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:23.047994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b8d4-8224-4dc8-8d67-4a1b649d8598 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:23.048277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b8d4-8224-4dc8-8d67-4a1b649d8598 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:23.060101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9b8d4-8224-4dc8-8d67-4a1b649d8598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:23.062378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9b8d4-8224-4dc8-8d67-4a1b649d8598 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:23.088017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 910/1826] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:23.092416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dfd0b-bef7-4f11-a6a1-dd9697e13dd4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:23.103318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dfd0b-bef7-4f11-a6a1-dd9697e13dd4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:23.108913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28002-5434-4ef2-bbc3-e7eb50318e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:23.109806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28002-5434-4ef2-bbc3-e7eb50318e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:23.110391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28002-5434-4ef2-bbc3-e7eb50318e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:23.110391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28002-5434-4ef2-bbc3-e7eb50318e6e admin admin] PATH_INFO: `/v3/projects/f33fcf7bb7a044faba875c74e11175cf/users/e88bfb6626034bdbbe321378ac1d5eb2/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:23.128434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f28002-5434-4ef2-bbc3-e7eb50318e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33fcf7bb7a044faba875c74e11175cf, user_id=e88bfb6626034bdbbe321378ac1d5eb2, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:23.131146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f28002-5434-4ef2-bbc3-e7eb50318e6e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:23.214118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 917/1827] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:23 2022] PUT /identity/v3/projects/f33fcf7bb7a044faba875c74e11175cf/users/e88bfb6626034bdbbe321378ac1d5eb2/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:23.227716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645b9f-607a-451d-9daf-3d7bd63cf021 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:23.228127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645b9f-607a-451d-9daf-3d7bd63cf021 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:23.228486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41645b9f-607a-451d-9daf-3d7bd63cf021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:23.270770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-41645b9f-607a-451d-9daf-3d7bd63cf021 None None] MFA Rules not processed for user `e88bfb6626034bdbbe321378ac1d5eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:23.271992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-41645b9f-607a-451d-9daf-3d7bd63cf021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:23.365146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 911/1828] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:23 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:23.368144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28002-5434-4ef2-bbc3-e7eb50318e6e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:23.446693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:23.483850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28002-5434-4ef2-bbc3-e7eb50318e6e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:23.501621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43cda8-0fec-4602-a869-cb5e80241cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:23.502306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43cda8-0fec-4602-a869-cb5e80241cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:23.502772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43cda8-0fec-4602-a869-cb5e80241cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:23.503055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43cda8-0fec-4602-a869-cb5e80241cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:23.503782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43cda8-0fec-4602-a869-cb5e80241cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:23.505886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43cda8-0fec-4602-a869-cb5e80241cb1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:23.621904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 918/1829] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:23 2022] POST /identity/v3/projects => generated 363 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:23.626158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41645b9f-607a-451d-9daf-3d7bd63cf021 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:23.637854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41645b9f-607a-451d-9daf-3d7bd63cf021 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:23.648266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:23.709663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5902a52-94fa-420b-a947-5c9f5cc9990d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:23.710905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5902a52-94fa-420b-a947-5c9f5cc9990d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:23.710905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5902a52-94fa-420b-a947-5c9f5cc9990d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:23.710905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5902a52-94fa-420b-a947-5c9f5cc9990d admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:23.711701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5902a52-94fa-420b-a947-5c9f5cc9990d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:23.713663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5902a52-94fa-420b-a947-5c9f5cc9990d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:23.818750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 912/1830] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:23 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:23.826966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43cda8-0fec-4602-a869-cb5e80241cb1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:23.837974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43cda8-0fec-4602-a869-cb5e80241cb1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:23.843393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291c79e-6631-4bbc-b3cf-78d5a2f4e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:23.844052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291c79e-6631-4bbc-b3cf-78d5a2f4e3bc admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:23.844436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291c79e-6631-4bbc-b3cf-78d5a2f4e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:23.844614 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291c79e-6631-4bbc-b3cf-78d5a2f4e3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:23.845341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291c79e-6631-4bbc-b3cf-78d5a2f4e3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:23.847030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291c79e-6631-4bbc-b3cf-78d5a2f4e3bc admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:23.859082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 919/1831] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:23.863330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5902a52-94fa-420b-a947-5c9f5cc9990d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:23.874037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5902a52-94fa-420b-a947-5c9f5cc9990d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:23.878539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8aa2-3e71-40af-b2e5-5cad0bc37b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:23.879458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8aa2-3e71-40af-b2e5-5cad0bc37b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:23.879747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8aa2-3e71-40af-b2e5-5cad0bc37b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:23.880026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df8aa2-3e71-40af-b2e5-5cad0bc37b5d admin admin] PATH_INFO: `/v3/projects/8afe70ea3d3845d19cec376f31236eb9/users/b5e620e6aab74e8686636bf09b5ff836/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:23.896340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df8aa2-3e71-40af-b2e5-5cad0bc37b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afe70ea3d3845d19cec376f31236eb9, user_id=b5e620e6aab74e8686636bf09b5ff836, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:23.899096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df8aa2-3e71-40af-b2e5-5cad0bc37b5d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:24.011221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 913/1832] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:23 2022] PUT /identity/v3/projects/8afe70ea3d3845d19cec376f31236eb9/users/b5e620e6aab74e8686636bf09b5ff836/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:24.015426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291c79e-6631-4bbc-b3cf-78d5a2f4e3bc admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:24.025575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291c79e-6631-4bbc-b3cf-78d5a2f4e3bc admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:24.083931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06777d4a-155b-4955-b732-4f2d773b7d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:24.084733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06777d4a-155b-4955-b732-4f2d773b7d76 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:24.085088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06777d4a-155b-4955-b732-4f2d773b7d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:24.085430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06777d4a-155b-4955-b732-4f2d773b7d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:24.086290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06777d4a-155b-4955-b732-4f2d773b7d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:24.087990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06777d4a-155b-4955-b732-4f2d773b7d76 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:24.103420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 920/1833] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:24.104650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8aa2-3e71-40af-b2e5-5cad0bc37b5d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:24.117921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df8aa2-3e71-40af-b2e5-5cad0bc37b5d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:24.122852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:24.124402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:24.124638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:24.124915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] PATH_INFO: `/v3/projects/8afe70ea3d3845d19cec376f31236eb9/users/b5e620e6aab74e8686636bf09b5ff836/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:24.126914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afe70ea3d3845d19cec376f31236eb9, user_id=b5e620e6aab74e8686636bf09b5ff836, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:24.129649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:24.140422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 914/1834] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:24 2022] PUT /identity/v3/projects/8afe70ea3d3845d19cec376f31236eb9/users/b5e620e6aab74e8686636bf09b5ff836/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:24.150679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391834b-ef61-4d59-876c-8ade45b34bdf None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:24.150970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391834b-ef61-4d59-876c-8ade45b34bdf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:24.151253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391834b-ef61-4d59-876c-8ade45b34bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:24.178059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4391834b-ef61-4d59-876c-8ade45b34bdf None None] MFA Rules not processed for user `b5e620e6aab74e8686636bf09b5ff836`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:24.179277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4391834b-ef61-4d59-876c-8ade45b34bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:24.252671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 921/1835] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:24 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:24.265488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:24.267193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:24.323735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061de92f-c2fa-493a-b957-d8bfec142dcd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:24.327408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ca27-92b8-4a16-a68b-0bd0e80c95da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:24.328057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ca27-92b8-4a16-a68b-0bd0e80c95da service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:24.328340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ca27-92b8-4a16-a68b-0bd0e80c95da service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:24.328615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ca27-92b8-4a16-a68b-0bd0e80c95da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:24.337837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ca27-92b8-4a16-a68b-0bd0e80c95da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:24.339557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ca27-92b8-4a16-a68b-0bd0e80c95da service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:24.349815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 915/1836] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:25.438120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391834b-ef61-4d59-876c-8ade45b34bdf None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:25.485402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:25.500135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391834b-ef61-4d59-876c-8ade45b34bdf None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:25.504667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:25.505535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:25.505823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:25.506106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] PATH_INFO: `/v3/users/e88bfb6626034bdbbe321378ac1d5eb2` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:25.507287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e88bfb6626034bdbbe321378ac1d5eb2)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:25.509253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:25.727227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e88bfb6626034bdbbe321378ac1d5eb2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:25.739682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e88bfb6626034bdbbe321378ac1d5eb2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:25.740325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e88bfb6626034bdbbe321378ac1d5eb2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:25.740943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e88bfb6626034bdbbe321378ac1d5eb2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:25.750141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e88bfb6626034bdbbe321378ac1d5eb2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:25.854160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 922/1837] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:25 2022] DELETE /identity/v3/users/e88bfb6626034bdbbe321378ac1d5eb2 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:25.860035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34afd-982d-4271-9c33-bac334d8ea62 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:25.860497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34afd-982d-4271-9c33-bac334d8ea62 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:25.860627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34afd-982d-4271-9c33-bac334d8ea62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:25.878063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-90d34afd-982d-4271-9c33-bac334d8ea62 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:25.879452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-90d34afd-982d-4271-9c33-bac334d8ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:25.936967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 916/1838] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:25.946765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:25.948187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:26.015092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abcefc-9134-408e-a5db-7672d154e2f9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:26.020001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cb76a-8243-4c9e-9e7d-a749a63e7c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:26.020001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb76a-8243-4c9e-9e7d-a749a63e7c7d service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:26.020001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb76a-8243-4c9e-9e7d-a749a63e7c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:26.020229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb76a-8243-4c9e-9e7d-a749a63e7c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:26.030541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cb76a-8243-4c9e-9e7d-a749a63e7c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:26.032175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cb76a-8243-4c9e-9e7d-a749a63e7c7d service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:26.045104 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 923/1839] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:26.583911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d34afd-982d-4271-9c33-bac334d8ea62 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:26.585412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-90d34afd-982d-4271-9c33-bac334d8ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:26.598770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d34afd-982d-4271-9c33-bac334d8ea62 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:26.603406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:26.605843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:26.605843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:26.605843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] PATH_INFO: `/v3/projects/f33fcf7bb7a044faba875c74e11175cf` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:26.606090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33fcf7bb7a044faba875c74e11175cf)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:26.608083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:26.609770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:26.723386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33fcf7bb7a044faba875c74e11175cf', 'request_id': 'req-b68595c1-b694-4991-9798-dd4633b30a42'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:26.729525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33fcf7bb7a044faba875c74e11175cf', 'request_id': 'req-b68595c1-b694-4991-9798-dd4633b30a42'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:26.735474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33fcf7bb7a044faba875c74e11175cf', 'request_id': 'req-b68595c1-b694-4991-9798-dd4633b30a42'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:26.838318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] The token cache is being invalidate because project f33fcf7bb7a044faba875c74e11175cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:26.838538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:26.839751 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:26.841394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 917/1840] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:26 2022] DELETE /identity/v3/projects/f33fcf7bb7a044faba875c74e11175cf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:26.845578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cb76a-8243-4c9e-9e7d-a749a63e7c7d service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:26.855654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cb76a-8243-4c9e-9e7d-a749a63e7c7d service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:26.913180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:26.914088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:26.914372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:26.914677 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] PATH_INFO: `/v3/users/b5e620e6aab74e8686636bf09b5ff836` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:26.915906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e620e6aab74e8686636bf09b5ff836)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:26.917822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:26.929063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:27.138677 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e620e6aab74e8686636bf09b5ff836'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:27.151649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e620e6aab74e8686636bf09b5ff836'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:27.152332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e620e6aab74e8686636bf09b5ff836'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:27.152965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e620e6aab74e8686636bf09b5ff836'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:27.160320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a80582da-f45e-4bfb-9fbd-e4ad888275a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e620e6aab74e8686636bf09b5ff836'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:27.239494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 924/1841] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:26 2022] DELETE /identity/v3/users/b5e620e6aab74e8686636bf09b5ff836 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:27.921132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:27.922596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:27.937581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68595c1-b694-4991-9798-dd4633b30a42 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:27.996410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:27.997289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:27.997580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:27.997862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] PATH_INFO: `/v3/projects/8afe70ea3d3845d19cec376f31236eb9` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:27.999062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8afe70ea3d3845d19cec376f31236eb9)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:28.001144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:28.149057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8afe70ea3d3845d19cec376f31236eb9', 'request_id': 'req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:28.156043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8afe70ea3d3845d19cec376f31236eb9', 'request_id': 'req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:28.161787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8afe70ea3d3845d19cec376f31236eb9', 'request_id': 'req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:28.302016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] The token cache is being invalidate because project 8afe70ea3d3845d19cec376f31236eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:28.302338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:28.302970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4970dbb4-cf59-4e75-813f-d0a8cf1fac06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:28.304559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 918/1842] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:27 2022] DELETE /identity/v3/projects/8afe70ea3d3845d19cec376f31236eb9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:28.309367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d109-2fc5-4849-aa00-73e6c808fe91 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:28.309803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d109-2fc5-4849-aa00-73e6c808fe91 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:28.310013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792d109-2fc5-4849-aa00-73e6c808fe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:28.328192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1792d109-2fc5-4849-aa00-73e6c808fe91 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:28.329390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1792d109-2fc5-4849-aa00-73e6c808fe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:28.390941 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 925/1843] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:28.398963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048e72b-5e6a-48ac-beff-7c935fb17db6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:28.399260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048e72b-5e6a-48ac-beff-7c935fb17db6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:28.399580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048e72b-5e6a-48ac-beff-7c935fb17db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:28.420785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3048e72b-5e6a-48ac-beff-7c935fb17db6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:28.421923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3048e72b-5e6a-48ac-beff-7c935fb17db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:28.432238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 919/1844] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:28.436168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792d109-2fc5-4849-aa00-73e6c808fe91 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:28.445707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792d109-2fc5-4849-aa00-73e6c808fe91 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:28.450551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:28.451756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:28.452076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:28.452680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:28.453081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:28.453393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:28.455531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:28.469290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 926/1845] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:54:28.473869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048e72b-5e6a-48ac-beff-7c935fb17db6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:28.483349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048e72b-5e6a-48ac-beff-7c935fb17db6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:28.491947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50b20-3441-4bf4-bc49-dbb4ccebadea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:28.492837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50b20-3441-4bf4-bc49-dbb4ccebadea admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:28.493274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50b20-3441-4bf4-bc49-dbb4ccebadea admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:28.493702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50b20-3441-4bf4-bc49-dbb4ccebadea admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:28.494615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50b20-3441-4bf4-bc49-dbb4ccebadea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:28.502704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50b20-3441-4bf4-bc49-dbb4ccebadea admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:28.513296 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:28.617897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 920/1846] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:28 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:28.621872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:28.631809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db09474-35fa-4207-883d-f865d6ef85d6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:28.689005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fde44-9956-4006-aafe-fd873b499a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:28.689584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fde44-9956-4006-aafe-fd873b499a88 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:28.689877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fde44-9956-4006-aafe-fd873b499a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:28.690166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fde44-9956-4006-aafe-fd873b499a88 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:28.690861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fde44-9956-4006-aafe-fd873b499a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:28.692816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fde44-9956-4006-aafe-fd873b499a88 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:28.874242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 927/1847] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:28 2022] POST /identity/v3/users => generated 295 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:54:28.878289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50b20-3441-4bf4-bc49-dbb4ccebadea admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:28.888777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50b20-3441-4bf4-bc49-dbb4ccebadea admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:28.894353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ba720-3555-4617-a4e6-373d75791d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:28.895066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ba720-3555-4617-a4e6-373d75791d25 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:28.895396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ba720-3555-4617-a4e6-373d75791d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:28.895733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ba720-3555-4617-a4e6-373d75791d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:28.896469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ba720-3555-4617-a4e6-373d75791d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:28.898115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ba720-3555-4617-a4e6-373d75791d25 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:28.912114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 921/1848] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:28.913274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fde44-9956-4006-aafe-fd873b499a88 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:28.919594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:28.924170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fde44-9956-4006-aafe-fd873b499a88 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:28.928718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f87c4-41a0-4b04-b918-4e68eb3afc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:28.929377 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f87c4-41a0-4b04-b918-4e68eb3afc90 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:28.929667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f87c4-41a0-4b04-b918-4e68eb3afc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:28.929980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f87c4-41a0-4b04-b918-4e68eb3afc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:28.930692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f87c4-41a0-4b04-b918-4e68eb3afc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:28.932326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f87c4-41a0-4b04-b918-4e68eb3afc90 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:28.942837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 928/1849] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:54:28.947074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ba720-3555-4617-a4e6-373d75791d25 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:28.957029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ba720-3555-4617-a4e6-373d75791d25 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:28.961577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2062-5e54-4a3d-8860-d2847d29dc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:28.962451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2062-5e54-4a3d-8860-d2847d29dc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:28.962739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2062-5e54-4a3d-8860-d2847d29dc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:28.963019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2062-5e54-4a3d-8860-d2847d29dc0f admin admin] PATH_INFO: `/v3/projects/ed6d67f2c4f84a619936cae8d1a74c4e/users/b805f20678d04e818769a07f13ed7625/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:28.978487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f2062-5e54-4a3d-8860-d2847d29dc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed6d67f2c4f84a619936cae8d1a74c4e, user_id=b805f20678d04e818769a07f13ed7625, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:28.981220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f2062-5e54-4a3d-8860-d2847d29dc0f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:29.080360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 922/1850] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:28 2022] PUT /identity/v3/projects/ed6d67f2c4f84a619936cae8d1a74c4e/users/b805f20678d04e818769a07f13ed7625/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:29.091264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2180cd5-aa78-4027-8dbb-af29c9b7a109 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:29.091611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2180cd5-aa78-4027-8dbb-af29c9b7a109 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:29.091919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2180cd5-aa78-4027-8dbb-af29c9b7a109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:29.125493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f2180cd5-aa78-4027-8dbb-af29c9b7a109 None None] MFA Rules not processed for user `b805f20678d04e818769a07f13ed7625`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:29.126859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f2180cd5-aa78-4027-8dbb-af29c9b7a109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:29.190684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 929/1851] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:29.195120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2062-5e54-4a3d-8860-d2847d29dc0f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:29.265550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2062-5e54-4a3d-8860-d2847d29dc0f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:29.273805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd131a63-b295-49cf-9f71-7d4a8dc78bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:29.274484 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131a63-b295-49cf-9f71-7d4a8dc78bcf admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:29.274772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131a63-b295-49cf-9f71-7d4a8dc78bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:29.275052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131a63-b295-49cf-9f71-7d4a8dc78bcf admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:29.275792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd131a63-b295-49cf-9f71-7d4a8dc78bcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:29.277718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd131a63-b295-49cf-9f71-7d4a8dc78bcf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:29.449423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 923/1852] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:29 2022] POST /identity/v3/projects => generated 387 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:29.452039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2180cd5-aa78-4027-8dbb-af29c9b7a109 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:29.463857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2180cd5-aa78-4027-8dbb-af29c9b7a109 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:29.521961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73eb2-811a-4153-90d4-1f5d319fdfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:29.522644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73eb2-811a-4153-90d4-1f5d319fdfe2 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:29.522928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73eb2-811a-4153-90d4-1f5d319fdfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:29.523359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73eb2-811a-4153-90d4-1f5d319fdfe2 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:29.524061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a73eb2-811a-4153-90d4-1f5d319fdfe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:29.525991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a73eb2-811a-4153-90d4-1f5d319fdfe2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:29.646827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 930/1853] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:29 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:29.650684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd131a63-b295-49cf-9f71-7d4a8dc78bcf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:29.661482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd131a63-b295-49cf-9f71-7d4a8dc78bcf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:29.666818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704a066-2ff3-40e8-8334-5c9bf08dcdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:29.667727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704a066-2ff3-40e8-8334-5c9bf08dcdca admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:29.668017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704a066-2ff3-40e8-8334-5c9bf08dcdca admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:29.668297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704a066-2ff3-40e8-8334-5c9bf08dcdca admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:29.669251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704a066-2ff3-40e8-8334-5c9bf08dcdca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:29.671485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704a066-2ff3-40e8-8334-5c9bf08dcdca admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:29.687335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 924/1854] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:29.688516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73eb2-811a-4153-90d4-1f5d319fdfe2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:29.700364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73eb2-811a-4153-90d4-1f5d319fdfe2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:29.706258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08d486-b5ad-48a0-828b-529af34c6d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:29.707439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08d486-b5ad-48a0-828b-529af34c6d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:29.707554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08d486-b5ad-48a0-828b-529af34c6d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:29.707752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08d486-b5ad-48a0-828b-529af34c6d64 admin admin] PATH_INFO: `/v3/projects/81146abeb6f64098bb6319f7d0c713b6/users/b26810c2de4b4d3980886f136dfb6d4c/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:29.729689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08d486-b5ad-48a0-828b-529af34c6d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81146abeb6f64098bb6319f7d0c713b6, user_id=b26810c2de4b4d3980886f136dfb6d4c, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:29.732453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08d486-b5ad-48a0-828b-529af34c6d64 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:29.805533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 931/1855] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:29 2022] PUT /identity/v3/projects/81146abeb6f64098bb6319f7d0c713b6/users/b26810c2de4b4d3980886f136dfb6d4c/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:29.809859 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704a066-2ff3-40e8-8334-5c9bf08dcdca admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:29.822421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704a066-2ff3-40e8-8334-5c9bf08dcdca admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:29.879824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26761c0b-f02a-40dd-8f36-c392b31dbe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:29.880501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26761c0b-f02a-40dd-8f36-c392b31dbe64 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:29.880915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26761c0b-f02a-40dd-8f36-c392b31dbe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:29.881098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26761c0b-f02a-40dd-8f36-c392b31dbe64 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:29.881817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26761c0b-f02a-40dd-8f36-c392b31dbe64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:29.883499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26761c0b-f02a-40dd-8f36-c392b31dbe64 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:29.893416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 925/1856] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:29.897610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08d486-b5ad-48a0-828b-529af34c6d64 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:29.907930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08d486-b5ad-48a0-828b-529af34c6d64 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:29.912588 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:29.913471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:29.913755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:29.914035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] PATH_INFO: `/v3/projects/81146abeb6f64098bb6319f7d0c713b6/users/b26810c2de4b4d3980886f136dfb6d4c/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:29.916017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81146abeb6f64098bb6319f7d0c713b6, user_id=b26810c2de4b4d3980886f136dfb6d4c, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:29.918719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:29.930086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 932/1857] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:29 2022] PUT /identity/v3/projects/81146abeb6f64098bb6319f7d0c713b6/users/b26810c2de4b4d3980886f136dfb6d4c/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:29.941939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7144-e692-486d-bace-8ba11576b473 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:29.942234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7144-e692-486d-bace-8ba11576b473 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:29.942520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7144-e692-486d-bace-8ba11576b473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:29.969509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f83e7144-e692-486d-bace-8ba11576b473 None None] MFA Rules not processed for user `b26810c2de4b4d3980886f136dfb6d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:29.970812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f83e7144-e692-486d-bace-8ba11576b473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:29.985196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:30.041243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 926/1858] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:30.052614 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:30.054485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:30.103510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:30.120340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fc84e-bf62-4a4a-8ca4-5d857bad8f99 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:30.124036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dbf03-fcae-4400-b2f1-7d01cfb4867a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:30.124690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dbf03-fcae-4400-b2f1-7d01cfb4867a service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:30.124979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dbf03-fcae-4400-b2f1-7d01cfb4867a service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:30.125341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dbf03-fcae-4400-b2f1-7d01cfb4867a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:30.134396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dbf03-fcae-4400-b2f1-7d01cfb4867a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:30.135936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dbf03-fcae-4400-b2f1-7d01cfb4867a service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:30.146009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 933/1859] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:30.278292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e7144-e692-486d-bace-8ba11576b473 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:30.342975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e7144-e692-486d-bace-8ba11576b473 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:30.347638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:30.348517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:30.348802 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:30.349081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] PATH_INFO: `/v3/users/b805f20678d04e818769a07f13ed7625` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:30.350290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b805f20678d04e818769a07f13ed7625)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:30.352287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:30.571572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b805f20678d04e818769a07f13ed7625'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:30.583476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b805f20678d04e818769a07f13ed7625'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:30.584171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b805f20678d04e818769a07f13ed7625'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:30.584792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b805f20678d04e818769a07f13ed7625'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:30.592259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b805f20678d04e818769a07f13ed7625'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:30.689356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 927/1860] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:30 2022] DELETE /identity/v3/users/b805f20678d04e818769a07f13ed7625 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:30.694094 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00113a34-7250-4d9a-a708-c3fb0491eeb1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:30.694580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00113a34-7250-4d9a-a708-c3fb0491eeb1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:30.694668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00113a34-7250-4d9a-a708-c3fb0491eeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:30.711511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-00113a34-7250-4d9a-a708-c3fb0491eeb1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:30.712719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-00113a34-7250-4d9a-a708-c3fb0491eeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:30.776256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 934/1861] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:30.784562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:30.786044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:30.831555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:30.867688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ed24e-7f9a-4d13-bd91-6a4accff2b80 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:30.872217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fadc-ab4a-4051-8ec5-a0c20dbeaad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:30.879316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fadc-ab4a-4051-8ec5-a0c20dbeaad7 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:30.879704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fadc-ab4a-4051-8ec5-a0c20dbeaad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:30.879995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fadc-ab4a-4051-8ec5-a0c20dbeaad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:30.898075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387fadc-ab4a-4051-8ec5-a0c20dbeaad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:30.899655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387fadc-ab4a-4051-8ec5-a0c20dbeaad7 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:30.911194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 928/1862] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:31.458551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00113a34-7250-4d9a-a708-c3fb0491eeb1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:31.460021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-00113a34-7250-4d9a-a708-c3fb0491eeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:31.475050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00113a34-7250-4d9a-a708-c3fb0491eeb1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:31.479704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:31.480577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:31.480863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:31.481143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] PATH_INFO: `/v3/projects/ed6d67f2c4f84a619936cae8d1a74c4e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:31.482380 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed6d67f2c4f84a619936cae8d1a74c4e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:31.485388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:31.652798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed6d67f2c4f84a619936cae8d1a74c4e', 'request_id': 'req-62273478-3207-4a33-b2ab-526567468423'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:31.658961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed6d67f2c4f84a619936cae8d1a74c4e', 'request_id': 'req-62273478-3207-4a33-b2ab-526567468423'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:31.770865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed6d67f2c4f84a619936cae8d1a74c4e', 'request_id': 'req-62273478-3207-4a33-b2ab-526567468423'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:31.782489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] The token cache is being invalidate because project ed6d67f2c4f84a619936cae8d1a74c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:31.782773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:31.783439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:31.784990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 935/1863] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:31 2022] DELETE /identity/v3/projects/ed6d67f2c4f84a619936cae8d1a74c4e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:31.789178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fadc-ab4a-4051-8ec5-a0c20dbeaad7 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:31.799024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fadc-ab4a-4051-8ec5-a0c20dbeaad7 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:31.855493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:31.856378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:31.856669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:31.856954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] PATH_INFO: `/v3/users/b26810c2de4b4d3980886f136dfb6d4c` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:31.858153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b26810c2de4b4d3980886f136dfb6d4c)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:31.860136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:32.095059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b26810c2de4b4d3980886f136dfb6d4c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:32.109390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b26810c2de4b4d3980886f136dfb6d4c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:32.110340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b26810c2de4b4d3980886f136dfb6d4c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:32.110962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b26810c2de4b4d3980886f136dfb6d4c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:32.122186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21a5e21a-1c13-47f2-a1af-2df220301c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b26810c2de4b4d3980886f136dfb6d4c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:32.228336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 929/1864] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:31 2022] DELETE /identity/v3/users/b26810c2de4b4d3980886f136dfb6d4c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:32.647032 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:32.648498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:32.663410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273478-3207-4a33-b2ab-526567468423 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:32.721872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:32.722829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:32.723127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:32.723452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] PATH_INFO: `/v3/projects/81146abeb6f64098bb6319f7d0c713b6` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:32.724649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81146abeb6f64098bb6319f7d0c713b6)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:32.726627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:32.729228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:32.939759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81146abeb6f64098bb6319f7d0c713b6', 'request_id': 'req-fdde2428-6776-43e9-b8f7-b87c7ce6f799'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:32.944799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81146abeb6f64098bb6319f7d0c713b6', 'request_id': 'req-fdde2428-6776-43e9-b8f7-b87c7ce6f799'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:33.045875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81146abeb6f64098bb6319f7d0c713b6', 'request_id': 'req-fdde2428-6776-43e9-b8f7-b87c7ce6f799'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:33.058162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] The token cache is being invalidate because project 81146abeb6f64098bb6319f7d0c713b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:33.058456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:33.059101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fdde2428-6776-43e9-b8f7-b87c7ce6f799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:33.060699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 936/1865] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:32 2022] DELETE /identity/v3/projects/81146abeb6f64098bb6319f7d0c713b6 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:33.066321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c37daf-ad87-41ac-9230-bb3f986d274d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.066790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c37daf-ad87-41ac-9230-bb3f986d274d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.066887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c37daf-ad87-41ac-9230-bb3f986d274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.085027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-14c37daf-ad87-41ac-9230-bb3f986d274d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:33.086254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-14c37daf-ad87-41ac-9230-bb3f986d274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:33.132925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:33.144926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 930/1866] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:33.152969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc245b-7a93-488e-be8a-de2665aece46 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.153264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc245b-7a93-488e-be8a-de2665aece46 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.153545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc245b-7a93-488e-be8a-de2665aece46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.176829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d5bc245b-7a93-488e-be8a-de2665aece46 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:33.178016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d5bc245b-7a93-488e-be8a-de2665aece46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:33.186892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 937/1867] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:33.191346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c37daf-ad87-41ac-9230-bb3f986d274d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:33.201032 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c37daf-ad87-41ac-9230-bb3f986d274d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:33.205912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:33.206575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.206866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.207149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.207908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:33.208224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:33.209853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:33.224650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 931/1868] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:54:33.235514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc245b-7a93-488e-be8a-de2665aece46 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:33.245230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc245b-7a93-488e-be8a-de2665aece46 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:33.253914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b513b8-7109-44bb-ab57-d595c2e93bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:33.254596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b513b8-7109-44bb-ab57-d595c2e93bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.254886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b513b8-7109-44bb-ab57-d595c2e93bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.255173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b513b8-7109-44bb-ab57-d595c2e93bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.255912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b513b8-7109-44bb-ab57-d595c2e93bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:33.257854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b513b8-7109-44bb-ab57-d595c2e93bf0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:33.369564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 938/1869] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:33 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:33.371065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:33.382556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b88773-a967-4059-8c31-629f29ef1f95 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:33.449117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48db3-92af-4338-b93f-481aff164370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:33.449828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48db3-92af-4338-b93f-481aff164370 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.450103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48db3-92af-4338-b93f-481aff164370 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.450379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48db3-92af-4338-b93f-481aff164370 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.451091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c48db3-92af-4338-b93f-481aff164370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:33.453651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c48db3-92af-4338-b93f-481aff164370 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:33.550108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 932/1870] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:33 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:33.553265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b513b8-7109-44bb-ab57-d595c2e93bf0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:33.563099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b513b8-7109-44bb-ab57-d595c2e93bf0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:33.567819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fade55-e6d1-4196-84d9-7d81e795766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:33.568491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fade55-e6d1-4196-84d9-7d81e795766e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.568781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fade55-e6d1-4196-84d9-7d81e795766e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.569065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fade55-e6d1-4196-84d9-7d81e795766e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.569832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fade55-e6d1-4196-84d9-7d81e795766e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:33.571470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fade55-e6d1-4196-84d9-7d81e795766e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:33.582751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 939/1871] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:33.586868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48db3-92af-4338-b93f-481aff164370 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:33.597421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48db3-92af-4338-b93f-481aff164370 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:33.602460 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaa303-165e-4cef-9524-16c582559b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:33.603400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa303-165e-4cef-9524-16c582559b3e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.604141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa303-165e-4cef-9524-16c582559b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.604141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaa303-165e-4cef-9524-16c582559b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.605035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaa303-165e-4cef-9524-16c582559b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:33.606854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaa303-165e-4cef-9524-16c582559b3e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:33.617902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 933/1872] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:33.622115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fade55-e6d1-4196-84d9-7d81e795766e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:33.633019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fade55-e6d1-4196-84d9-7d81e795766e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:33.637681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:33.638566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.638870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.639162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] PATH_INFO: `/v3/projects/8d7726f6c7444f16ac5ea9696eacdf57/users/a3efd91d94b44c6aa41087c2f017861f/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.655065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7726f6c7444f16ac5ea9696eacdf57, user_id=a3efd91d94b44c6aa41087c2f017861f, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:33.658077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:33.760533 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 940/1873] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:33 2022] PUT /identity/v3/projects/8d7726f6c7444f16ac5ea9696eacdf57/users/a3efd91d94b44c6aa41087c2f017861f/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:33.765750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbd0e5-e9c3-4218-9748-9c8ad8761638 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.773192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbd0e5-e9c3-4218-9748-9c8ad8761638 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.773192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbd0e5-e9c3-4218-9748-9c8ad8761638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.787894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-bfbbd0e5-e9c3-4218-9748-9c8ad8761638 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:33.789112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bfbbd0e5-e9c3-4218-9748-9c8ad8761638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:33.846342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 934/1874] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:33.855951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:33.858161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:33.915583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5ce06-1860-4fd5-904a-72c546de30a2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:33.919245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:33.919927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:33.920261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:33.921070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:33.930424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:33.931912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:33.941858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 941/1875] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:35.603610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a128b2b-7fcb-4dd7-a368-da89c1b06fb4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:35.604024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a128b2b-7fcb-4dd7-a368-da89c1b06fb4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:35.604181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a128b2b-7fcb-4dd7-a368-da89c1b06fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:35.635208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1a128b2b-7fcb-4dd7-a368-da89c1b06fb4 None None] MFA Rules not processed for user `a3efd91d94b44c6aa41087c2f017861f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:35.636600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1a128b2b-7fcb-4dd7-a368-da89c1b06fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:35.717536 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:35.742833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 935/1876] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:35.783438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:35.787059 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:35.839546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:35.866085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd422b-695c-4e6d-a0ce-bc78c66e9124 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:35.885443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:35.886665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:35.887113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:35.888458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:35.907999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:35.909692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:35.922210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 942/1877] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:38.903810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1bf97-55a0-4088-a35b-8a732e7e41bd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:38.904333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1bf97-55a0-4088-a35b-8a732e7e41bd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:38.904494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1bf97-55a0-4088-a35b-8a732e7e41bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:38.923086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d5a1bf97-55a0-4088-a35b-8a732e7e41bd None None] MFA Rules not processed for user `a3efd91d94b44c6aa41087c2f017861f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:38.924387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d5a1bf97-55a0-4088-a35b-8a732e7e41bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:38.931500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 936/1878] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:39.664582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:39.666183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:39.684084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6edc4-96c9-484b-9d29-0995d94beae3 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:39.690668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:39.691609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:39.691898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:39.692182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] PATH_INFO: `/v3/users/3a0064d22cee446792a5d2b65b6a9c06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:39.693421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0064d22cee446792a5d2b65b6a9c06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:39.695595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:39.985949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0064d22cee446792a5d2b65b6a9c06'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:39.997718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0064d22cee446792a5d2b65b6a9c06'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:39.998435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0064d22cee446792a5d2b65b6a9c06'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:39.999058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0064d22cee446792a5d2b65b6a9c06'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:40.006034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0064d22cee446792a5d2b65b6a9c06'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:40.093773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 943/1879] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:39 2022] DELETE /identity/v3/users/3a0064d22cee446792a5d2b65b6a9c06 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:40.101678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7a47e-6653-4a92-8f3b-757d8b98b026 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:40.102140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7a47e-6653-4a92-8f3b-757d8b98b026 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:40.102249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7a47e-6653-4a92-8f3b-757d8b98b026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:40.122182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-bef7a47e-6653-4a92-8f3b-757d8b98b026 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:40.123443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bef7a47e-6653-4a92-8f3b-757d8b98b026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:40.181872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 937/1880] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:40.191706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:40.193141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:40.251251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38a73-e5d6-45c8-aa02-4d92e85b5015 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:40.254887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc543a2-d94e-449c-8a9a-39d3a77a1b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:40.255570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc543a2-d94e-449c-8a9a-39d3a77a1b3e service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:40.255861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc543a2-d94e-449c-8a9a-39d3a77a1b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:40.256141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc543a2-d94e-449c-8a9a-39d3a77a1b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:40.265900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc543a2-d94e-449c-8a9a-39d3a77a1b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:40.267360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc543a2-d94e-449c-8a9a-39d3a77a1b3e service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:40.276829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 944/1881] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:40.892918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7a47e-6653-4a92-8f3b-757d8b98b026 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:40.894417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bef7a47e-6653-4a92-8f3b-757d8b98b026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:40.929475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7a47e-6653-4a92-8f3b-757d8b98b026 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:40.947659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:40.947659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:40.947659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:40.947659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] PATH_INFO: `/v3/projects/cb7682c725844eeaa1dd534cd74d3421` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:40.964282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7682c725844eeaa1dd534cd74d3421)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:40.966547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:41.189135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7682c725844eeaa1dd534cd74d3421', 'request_id': 'req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.196608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7682c725844eeaa1dd534cd74d3421', 'request_id': 'req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.204063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7682c725844eeaa1dd534cd74d3421', 'request_id': 'req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.290085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] The token cache is being invalidate because project cb7682c725844eeaa1dd534cd74d3421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:41.290555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.291194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.292968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 938/1882] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:40 2022] DELETE /identity/v3/projects/cb7682c725844eeaa1dd534cd74d3421 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:41.297558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc543a2-d94e-449c-8a9a-39d3a77a1b3e service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:41.307757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc543a2-d94e-449c-8a9a-39d3a77a1b3e service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:41.355182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:41.368970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:41.370066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:41.370360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:41.370643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] PATH_INFO: `/v3/users/8c346779743c42b4931c96b10346745a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:41.371862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c346779743c42b4931c96b10346745a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:41.374352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:41.560599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c346779743c42b4931c96b10346745a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.572664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c346779743c42b4931c96b10346745a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.573402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c346779743c42b4931c96b10346745a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.574051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c346779743c42b4931c96b10346745a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.586303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-873d69a5-2603-497a-a804-b3ff2044622f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c346779743c42b4931c96b10346745a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:41.672745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 945/1883] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:41 2022] DELETE /identity/v3/users/8c346779743c42b4931c96b10346745a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:42.652162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:42.652778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:42.677141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c6dad-ce83-4e37-b4fa-c3579eb85bcd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:42.763805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:42.765400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:42.765846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:42.766275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] PATH_INFO: `/v3/projects/af9239b3cc334441b79792dfd4308b6b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:42.767871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9239b3cc334441b79792dfd4308b6b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:42.771154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:42.784104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:42.917717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9239b3cc334441b79792dfd4308b6b', 'request_id': 'req-c1551d79-9c93-49df-a05c-8a6c642e5d33'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:42.939180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9239b3cc334441b79792dfd4308b6b', 'request_id': 'req-c1551d79-9c93-49df-a05c-8a6c642e5d33'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:42.952589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9239b3cc334441b79792dfd4308b6b', 'request_id': 'req-c1551d79-9c93-49df-a05c-8a6c642e5d33'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:43.072343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] The token cache is being invalidate because project af9239b3cc334441b79792dfd4308b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:43.073018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:43.074125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c1551d79-9c93-49df-a05c-8a6c642e5d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:43.080956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 939/1884] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:42 2022] DELETE /identity/v3/projects/af9239b3cc334441b79792dfd4308b6b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:43.082881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5da64-1e9e-41bd-a925-1a6b47d22bf6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:43.084169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5da64-1e9e-41bd-a925-1a6b47d22bf6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:43.084169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5da64-1e9e-41bd-a925-1a6b47d22bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:43.134531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-66a5da64-1e9e-41bd-a925-1a6b47d22bf6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:43.144100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-66a5da64-1e9e-41bd-a925-1a6b47d22bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:43.292989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 946/1885] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:43.293686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d82a-bed3-4896-9197-a4790d906bc1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:43.294085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d82a-bed3-4896-9197-a4790d906bc1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:43.294418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d82a-bed3-4896-9197-a4790d906bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:43.348287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b6e4d82a-bed3-4896-9197-a4790d906bc1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:43.351220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b6e4d82a-bed3-4896-9197-a4790d906bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:43.367105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 940/1886] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:54:43.371753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5da64-1e9e-41bd-a925-1a6b47d22bf6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:43.392417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5da64-1e9e-41bd-a925-1a6b47d22bf6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:43.412187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:43.413135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:43.413688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:43.414233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:43.415204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:43.415765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:43.417661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:43.439701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 947/1887] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:54:43.441508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d82a-bed3-4896-9197-a4790d906bc1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:43.451322 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d82a-bed3-4896-9197-a4790d906bc1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:43.459552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f9988-8fa6-48bd-a8c7-d8e9cf85b1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:43.460235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f9988-8fa6-48bd-a8c7-d8e9cf85b1ef admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:43.460551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f9988-8fa6-48bd-a8c7-d8e9cf85b1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:43.460846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f9988-8fa6-48bd-a8c7-d8e9cf85b1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:43.461753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f9988-8fa6-48bd-a8c7-d8e9cf85b1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:43.463774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f9988-8fa6-48bd-a8c7-d8e9cf85b1ef admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:43.595645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 941/1888] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:43 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:43.600039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:43.616515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74679d-21fa-4da5-bbe6-83162d8890b1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:43.699656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:43.707566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa7e5f-7ef5-495e-8b5d-059642ce9cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:43.708514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa7e5f-7ef5-495e-8b5d-059642ce9cea admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:43.708847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa7e5f-7ef5-495e-8b5d-059642ce9cea admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:43.709147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa7e5f-7ef5-495e-8b5d-059642ce9cea admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:43.709855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa7e5f-7ef5-495e-8b5d-059642ce9cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:43.712157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa7e5f-7ef5-495e-8b5d-059642ce9cea admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:43.812468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 948/1889] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:43 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:43.816271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f9988-8fa6-48bd-a8c7-d8e9cf85b1ef admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:43.827403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f9988-8fa6-48bd-a8c7-d8e9cf85b1ef admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:43.832154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301172e-c8ab-4efc-9e1f-0b59cf51f488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:43.832832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301172e-c8ab-4efc-9e1f-0b59cf51f488 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:43.833121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301172e-c8ab-4efc-9e1f-0b59cf51f488 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:43.833403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301172e-c8ab-4efc-9e1f-0b59cf51f488 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:43.834130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301172e-c8ab-4efc-9e1f-0b59cf51f488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:43.835800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301172e-c8ab-4efc-9e1f-0b59cf51f488 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:43.848172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 942/1890] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:43.853090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa7e5f-7ef5-495e-8b5d-059642ce9cea admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:43.863458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa7e5f-7ef5-495e-8b5d-059642ce9cea admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:43.868317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a225c5-75f3-43bb-bcbe-10c6fc19cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:43.869060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a225c5-75f3-43bb-bcbe-10c6fc19cad8 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:43.869478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a225c5-75f3-43bb-bcbe-10c6fc19cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:43.869817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a225c5-75f3-43bb-bcbe-10c6fc19cad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:43.870612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a225c5-75f3-43bb-bcbe-10c6fc19cad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:43.872309 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a225c5-75f3-43bb-bcbe-10c6fc19cad8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:43.883782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 949/1891] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:43.888189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301172e-c8ab-4efc-9e1f-0b59cf51f488 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:43.899519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301172e-c8ab-4efc-9e1f-0b59cf51f488 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:43.910515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837f524-7b3c-46e6-9f8f-88e1b5dbebb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:43.910515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f524-7b3c-46e6-9f8f-88e1b5dbebb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:43.910515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f524-7b3c-46e6-9f8f-88e1b5dbebb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:43.910515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f524-7b3c-46e6-9f8f-88e1b5dbebb6 admin admin] PATH_INFO: `/v3/projects/d7f848ab067d4d518ec7a1ff2844f372/users/077d195dd5df4b9fb71452b185ce99bb/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:43.926785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837f524-7b3c-46e6-9f8f-88e1b5dbebb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f848ab067d4d518ec7a1ff2844f372, user_id=077d195dd5df4b9fb71452b185ce99bb, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:43.929491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837f524-7b3c-46e6-9f8f-88e1b5dbebb6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:44.046516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 943/1892] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:43 2022] PUT /identity/v3/projects/d7f848ab067d4d518ec7a1ff2844f372/users/077d195dd5df4b9fb71452b185ce99bb/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:44.057547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7a5d3-2235-4d6a-b59c-529e98f245c8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:44.057843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7a5d3-2235-4d6a-b59c-529e98f245c8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:44.058125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7a5d3-2235-4d6a-b59c-529e98f245c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:44.085440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c4a7a5d3-2235-4d6a-b59c-529e98f245c8 None None] MFA Rules not processed for user `077d195dd5df4b9fb71452b185ce99bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:44.086683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c4a7a5d3-2235-4d6a-b59c-529e98f245c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:44.161566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 950/1893] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:44.165911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837f524-7b3c-46e6-9f8f-88e1b5dbebb6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:44.285480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837f524-7b3c-46e6-9f8f-88e1b5dbebb6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:44.296952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c0141-acb9-4d71-9e4d-f8c958af1aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:44.297648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c0141-acb9-4d71-9e4d-f8c958af1aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:44.297929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c0141-acb9-4d71-9e4d-f8c958af1aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:44.298208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c0141-acb9-4d71-9e4d-f8c958af1aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:44.298905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c0141-acb9-4d71-9e4d-f8c958af1aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:44.303500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c0141-acb9-4d71-9e4d-f8c958af1aa8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:44.402776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 944/1894] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:44 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:44.408003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7a5d3-2235-4d6a-b59c-529e98f245c8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:44.418976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7a5d3-2235-4d6a-b59c-529e98f245c8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:44.421362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0652d9c-39be-41e3-a502-bb6c0a3198e7 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:44.421673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0652d9c-39be-41e3-a502-bb6c0a3198e7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:44.421964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0652d9c-39be-41e3-a502-bb6c0a3198e7 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:44.423390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 945/1895] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:54:44.438704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f6b2-44e5-44dc-8fb3-d3da8144786a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:44.439049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f6b2-44e5-44dc-8fb3-d3da8144786a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:44.439476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f6b2-44e5-44dc-8fb3-d3da8144786a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:44.461246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1686f6b2-44e5-44dc-8fb3-d3da8144786a None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:44.462703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1686f6b2-44e5-44dc-8fb3-d3da8144786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:44.516931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:44.566698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599c32-239a-4fba-9b03-521f508bc3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:44.567711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599c32-239a-4fba-9b03-521f508bc3cc admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:44.568145 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599c32-239a-4fba-9b03-521f508bc3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:44.568570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599c32-239a-4fba-9b03-521f508bc3cc admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:44.569423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25599c32-239a-4fba-9b03-521f508bc3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:44.571654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25599c32-239a-4fba-9b03-521f508bc3cc admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:44.596519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 946/1896] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:54:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:54:44.609004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686f6b2-44e5-44dc-8fb3-d3da8144786a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:44.613076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1686f6b2-44e5-44dc-8fb3-d3da8144786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:44.654773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686f6b2-44e5-44dc-8fb3-d3da8144786a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:44.663310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac448bc8-a7ca-468f-b07b-af706ad21b5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:44.664148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac448bc8-a7ca-468f-b07b-af706ad21b5d service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:44.664521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac448bc8-a7ca-468f-b07b-af706ad21b5d service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:44.664906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac448bc8-a7ca-468f-b07b-af706ad21b5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:44.681002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac448bc8-a7ca-468f-b07b-af706ad21b5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:44.682607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 951/1897] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:44 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:44.683714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac448bc8-a7ca-468f-b07b-af706ad21b5d service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:44.688499 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599c32-239a-4fba-9b03-521f508bc3cc admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:44.706049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 947/1898] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:44.714624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599c32-239a-4fba-9b03-521f508bc3cc admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:44.729788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1acaaf-e77c-45c9-a203-075c7f469a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:44.733926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1acaaf-e77c-45c9-a203-075c7f469a75 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:44.733926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1acaaf-e77c-45c9-a203-075c7f469a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:44.733926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1acaaf-e77c-45c9-a203-075c7f469a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:44.734853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1acaaf-e77c-45c9-a203-075c7f469a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:44.736568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1acaaf-e77c-45c9-a203-075c7f469a75 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:44.762889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 952/1899] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:44.764716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac448bc8-a7ca-468f-b07b-af706ad21b5d service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:44.782481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac448bc8-a7ca-468f-b07b-af706ad21b5d service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:44.796423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac65c2c-1f80-488a-9104-31c1640e829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:44.797333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac65c2c-1f80-488a-9104-31c1640e829c admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:44.797620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac65c2c-1f80-488a-9104-31c1640e829c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:44.797901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac65c2c-1f80-488a-9104-31c1640e829c admin admin] PATH_INFO: `/v3/projects/9d6c1de2b4854f47a57a16b12937e934/users/b256e0895304403c8a354285fbc65d27/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:44.831871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac65c2c-1f80-488a-9104-31c1640e829c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6c1de2b4854f47a57a16b12937e934, user_id=b256e0895304403c8a354285fbc65d27, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:44.833199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac65c2c-1f80-488a-9104-31c1640e829c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:44.963147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 948/1900] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:44 2022] PUT /identity/v3/projects/9d6c1de2b4854f47a57a16b12937e934/users/b256e0895304403c8a354285fbc65d27/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:44.968378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1acaaf-e77c-45c9-a203-075c7f469a75 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:44.987766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1acaaf-e77c-45c9-a203-075c7f469a75 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:45.112525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:45.171348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2328f-47c0-4e2b-84e5-b1fdc5b9c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:45.172321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2328f-47c0-4e2b-84e5-b1fdc5b9c1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:45.177255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2328f-47c0-4e2b-84e5-b1fdc5b9c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:45.177255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2328f-47c0-4e2b-84e5-b1fdc5b9c1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:45.183452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2328f-47c0-4e2b-84e5-b1fdc5b9c1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:45.185153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2328f-47c0-4e2b-84e5-b1fdc5b9c1d7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:45.203206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 953/1901] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:45.210897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac65c2c-1f80-488a-9104-31c1640e829c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:45.230433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac65c2c-1f80-488a-9104-31c1640e829c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:45.268459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:45.272698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:45.275351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:45.277521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] PATH_INFO: `/v3/projects/9d6c1de2b4854f47a57a16b12937e934/users/b256e0895304403c8a354285fbc65d27/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:45.287938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6c1de2b4854f47a57a16b12937e934, user_id=b256e0895304403c8a354285fbc65d27, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:45.296141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:45.336480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 949/1902] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:45 2022] PUT /identity/v3/projects/9d6c1de2b4854f47a57a16b12937e934/users/b256e0895304403c8a354285fbc65d27/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:45.339257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b9d4-9877-46d5-b058-bcbbb41cc066 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:45.339257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b9d4-9877-46d5-b058-bcbbb41cc066 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:45.339752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b9d4-9877-46d5-b058-bcbbb41cc066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:45.452891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-37a1b9d4-9877-46d5-b058-bcbbb41cc066 None None] MFA Rules not processed for user `b256e0895304403c8a354285fbc65d27`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:45.458642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-37a1b9d4-9877-46d5-b058-bcbbb41cc066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:45.620110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 954/1903] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:45.643954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:45.643954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:45.795875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912654c2-06d8-479f-94c5-59082543953a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:45.802381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c662a00-8942-4240-aaf1-bb766610cde1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:45.802381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c662a00-8942-4240-aaf1-bb766610cde1 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:45.802381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c662a00-8942-4240-aaf1-bb766610cde1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:45.802381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c662a00-8942-4240-aaf1-bb766610cde1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:45.813567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c662a00-8942-4240-aaf1-bb766610cde1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:45.815177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c662a00-8942-4240-aaf1-bb766610cde1 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:45.826328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 950/1904] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:45.955966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b9d4-9877-46d5-b058-bcbbb41cc066 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:46.042645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b9d4-9877-46d5-b058-bcbbb41cc066 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:46.050900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:46.051973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:46.052333 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:46.052691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] PATH_INFO: `/v3/users/077d195dd5df4b9fb71452b185ce99bb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:46.054049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=077d195dd5df4b9fb71452b185ce99bb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:46.057351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:46.532893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077d195dd5df4b9fb71452b185ce99bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:46.544642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077d195dd5df4b9fb71452b185ce99bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:46.545342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077d195dd5df4b9fb71452b185ce99bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:46.545963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077d195dd5df4b9fb71452b185ce99bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:46.547171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:46.553135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077d195dd5df4b9fb71452b185ce99bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:46.683836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 955/1905] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:45 2022] DELETE /identity/v3/users/077d195dd5df4b9fb71452b185ce99bb => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:46.688702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e866c-58b9-43ab-9350-15ab8d18efce None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:46.689197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e866c-58b9-43ab-9350-15ab8d18efce None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:46.689274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e866c-58b9-43ab-9350-15ab8d18efce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:46.710388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b73e866c-58b9-43ab-9350-15ab8d18efce None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:46.711854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b73e866c-58b9-43ab-9350-15ab8d18efce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:46.795047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 951/1906] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:46.805169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:46.806795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:46.870733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6b3ab-4b45-447f-b859-bc923328c644 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:46.875728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7b29-e9ff-42c5-a841-142d5b5682ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:46.876431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7b29-e9ff-42c5-a841-142d5b5682ed service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:46.876718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7b29-e9ff-42c5-a841-142d5b5682ed service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:46.876996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7b29-e9ff-42c5-a841-142d5b5682ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:46.887316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc7b29-e9ff-42c5-a841-142d5b5682ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:46.888872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc7b29-e9ff-42c5-a841-142d5b5682ed service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:46.901332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 956/1907] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:47.392646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e866c-58b9-43ab-9350-15ab8d18efce None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:47.394247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b73e866c-58b9-43ab-9350-15ab8d18efce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:47.402690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:47.418890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e866c-58b9-43ab-9350-15ab8d18efce None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:47.424424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:47.425355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:47.425648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:47.425929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] PATH_INFO: `/v3/projects/d7f848ab067d4d518ec7a1ff2844f372` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:47.427722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f848ab067d4d518ec7a1ff2844f372)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:47.429659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:47.581543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f848ab067d4d518ec7a1ff2844f372', 'request_id': 'req-a24dd253-5702-496e-82fc-eed85e3fbafa'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:47.588811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f848ab067d4d518ec7a1ff2844f372', 'request_id': 'req-a24dd253-5702-496e-82fc-eed85e3fbafa'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:47.595155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f848ab067d4d518ec7a1ff2844f372', 'request_id': 'req-a24dd253-5702-496e-82fc-eed85e3fbafa'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:47.761014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] The token cache is being invalidate because project d7f848ab067d4d518ec7a1ff2844f372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:47.761871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:47.762808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:47.766924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 952/1908] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:47 2022] DELETE /identity/v3/projects/d7f848ab067d4d518ec7a1ff2844f372 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:47.769498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7b29-e9ff-42c5-a841-142d5b5682ed service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:47.779995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7b29-e9ff-42c5-a841-142d5b5682ed service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:47.851700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:47.852872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:47.853317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:47.853777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] PATH_INFO: `/v3/users/b256e0895304403c8a354285fbc65d27` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:47.855365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b256e0895304403c8a354285fbc65d27)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:47.858339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:48.126283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b256e0895304403c8a354285fbc65d27'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:48.138749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b256e0895304403c8a354285fbc65d27'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:48.144572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b256e0895304403c8a354285fbc65d27'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:48.144572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b256e0895304403c8a354285fbc65d27'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:48.151560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0d9dcfc6-d2d2-461c-b933-12b905244ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b256e0895304403c8a354285fbc65d27'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:48.268919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 957/1909] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:47 2022] DELETE /identity/v3/users/b256e0895304403c8a354285fbc65d27 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:48.823781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:48.825894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:48.847006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dd253-5702-496e-82fc-eed85e3fbafa admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:48.905427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:48.906323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:48.906607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:48.906884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] PATH_INFO: `/v3/projects/9d6c1de2b4854f47a57a16b12937e934` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:48.908164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6c1de2b4854f47a57a16b12937e934)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:48.910166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:49.021101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6c1de2b4854f47a57a16b12937e934', 'request_id': 'req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:49.027282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6c1de2b4854f47a57a16b12937e934', 'request_id': 'req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:49.035149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6c1de2b4854f47a57a16b12937e934', 'request_id': 'req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:49.226728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] The token cache is being invalidate because project 9d6c1de2b4854f47a57a16b12937e934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:49.227103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:49.227885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ee4a96bb-2caf-4f89-92e7-fda11b25ee63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:49.229631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 953/1910] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:48 2022] DELETE /identity/v3/projects/9d6c1de2b4854f47a57a16b12937e934 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:49.234580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569453b9-2db6-4a04-8891-e9e208558820 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.235062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569453b9-2db6-4a04-8891-e9e208558820 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.235171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569453b9-2db6-4a04-8891-e9e208558820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.243689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:49.255271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-569453b9-2db6-4a04-8891-e9e208558820 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:49.256539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-569453b9-2db6-4a04-8891-e9e208558820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:49.318840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 958/1911] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:54:49.327521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3739aec-8f46-435c-88b0-2a47f0fddecf None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.327819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3739aec-8f46-435c-88b0-2a47f0fddecf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.328126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3739aec-8f46-435c-88b0-2a47f0fddecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.348417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f3739aec-8f46-435c-88b0-2a47f0fddecf None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:49.349645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f3739aec-8f46-435c-88b0-2a47f0fddecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:49.358265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 954/1912] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:49.362562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569453b9-2db6-4a04-8891-e9e208558820 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:49.372255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569453b9-2db6-4a04-8891-e9e208558820 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:49.376939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:49.377606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.377896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.378178 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.379063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:49.384773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:49.384773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:49.405849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 959/1913] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:54:49.408367 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3739aec-8f46-435c-88b0-2a47f0fddecf None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:49.420591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3739aec-8f46-435c-88b0-2a47f0fddecf None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:49.426529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730d72e-68ad-49eb-9e5f-b122b4fb03d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:49.427226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d72e-68ad-49eb-9e5f-b122b4fb03d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.427548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d72e-68ad-49eb-9e5f-b122b4fb03d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.427829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d72e-68ad-49eb-9e5f-b122b4fb03d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.428528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730d72e-68ad-49eb-9e5f-b122b4fb03d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:49.430477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730d72e-68ad-49eb-9e5f-b122b4fb03d5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:49.435559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:49.543696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 955/1914] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:49 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:49.547719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:49.557585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84213fb-d414-4b9b-97c8-3127a3ac02c5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:49.613421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152935-b20f-4af4-9a1f-bc862b2bba24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:49.614093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152935-b20f-4af4-9a1f-bc862b2bba24 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.614385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152935-b20f-4af4-9a1f-bc862b2bba24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.614686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152935-b20f-4af4-9a1f-bc862b2bba24 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.615396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152935-b20f-4af4-9a1f-bc862b2bba24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:49.617319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152935-b20f-4af4-9a1f-bc862b2bba24 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:49.711765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 960/1915] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:49 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:49.716133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730d72e-68ad-49eb-9e5f-b122b4fb03d5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:49.726386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730d72e-68ad-49eb-9e5f-b122b4fb03d5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:49.730957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdf228-0916-4531-a3f3-827b8102e051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:49.731657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdf228-0916-4531-a3f3-827b8102e051 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.731948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdf228-0916-4531-a3f3-827b8102e051 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.732229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cdf228-0916-4531-a3f3-827b8102e051 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.732957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdf228-0916-4531-a3f3-827b8102e051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:49.734576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cdf228-0916-4531-a3f3-827b8102e051 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:49.745511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 956/1916] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:49.750300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152935-b20f-4af4-9a1f-bc862b2bba24 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:49.761158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152935-b20f-4af4-9a1f-bc862b2bba24 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:49.766692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb4c62-9393-4a7d-a1d9-40b22ce17643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:49.767434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb4c62-9393-4a7d-a1d9-40b22ce17643 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.767722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb4c62-9393-4a7d-a1d9-40b22ce17643 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.768000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb4c62-9393-4a7d-a1d9-40b22ce17643 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.768731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb4c62-9393-4a7d-a1d9-40b22ce17643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:49.770396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb4c62-9393-4a7d-a1d9-40b22ce17643 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:49.782311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 961/1917] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:49.787008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdf228-0916-4531-a3f3-827b8102e051 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:49.798253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cdf228-0916-4531-a3f3-827b8102e051 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:49.803145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00cd4c-e534-4acb-8d84-ef8ad6e88c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:49.804171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00cd4c-e534-4acb-8d84-ef8ad6e88c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.804460 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00cd4c-e534-4acb-8d84-ef8ad6e88c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.804742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00cd4c-e534-4acb-8d84-ef8ad6e88c27 admin admin] PATH_INFO: `/v3/projects/272a6838b51946adb8138c31ad203324/users/0a8068be6f8549d99ec8f329633905e8/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.821836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00cd4c-e534-4acb-8d84-ef8ad6e88c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272a6838b51946adb8138c31ad203324, user_id=0a8068be6f8549d99ec8f329633905e8, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:49.824721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00cd4c-e534-4acb-8d84-ef8ad6e88c27 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:49.930253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 957/1918] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:49 2022] PUT /identity/v3/projects/272a6838b51946adb8138c31ad203324/users/0a8068be6f8549d99ec8f329633905e8/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:49.941275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de1f0-132c-49c7-8148-ef5a9d2c647d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:49.941574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de1f0-132c-49c7-8148-ef5a9d2c647d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:49.941881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de1f0-132c-49c7-8148-ef5a9d2c647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:49.975159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-356de1f0-132c-49c7-8148-ef5a9d2c647d None None] MFA Rules not processed for user `0a8068be6f8549d99ec8f329633905e8`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:49.976435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-356de1f0-132c-49c7-8148-ef5a9d2c647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:50.041427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:50.045702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 962/1919] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:54:50.050611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00cd4c-e534-4acb-8d84-ef8ad6e88c27 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:50.120817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00cd4c-e534-4acb-8d84-ef8ad6e88c27 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:50.126188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd9f6c-0ec4-4ad8-9c44-e6a1fcfd7ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:50.126867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd9f6c-0ec4-4ad8-9c44-e6a1fcfd7ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:50.127153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd9f6c-0ec4-4ad8-9c44-e6a1fcfd7ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:50.127479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd9f6c-0ec4-4ad8-9c44-e6a1fcfd7ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:50.128176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd9f6c-0ec4-4ad8-9c44-e6a1fcfd7ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:50.130105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd9f6c-0ec4-4ad8-9c44-e6a1fcfd7ab6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:50.220051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 958/1920] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:50 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:54:50.223649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356de1f0-132c-49c7-8148-ef5a9d2c647d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:50.236729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356de1f0-132c-49c7-8148-ef5a9d2c647d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:50.320773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3866d-43ad-4992-a2f8-2c9ec000d55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:50.321469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3866d-43ad-4992-a2f8-2c9ec000d55e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:50.321758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3866d-43ad-4992-a2f8-2c9ec000d55e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:50.322038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3866d-43ad-4992-a2f8-2c9ec000d55e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:50.322728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3866d-43ad-4992-a2f8-2c9ec000d55e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:50.324686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3866d-43ad-4992-a2f8-2c9ec000d55e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:50.417259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 963/1921] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:50 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:50.421615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd9f6c-0ec4-4ad8-9c44-e6a1fcfd7ab6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:50.431918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd9f6c-0ec4-4ad8-9c44-e6a1fcfd7ab6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:50.436534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7193bb-9943-4944-8fbc-7ceabd963340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:50.437225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7193bb-9943-4944-8fbc-7ceabd963340 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:50.437524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7193bb-9943-4944-8fbc-7ceabd963340 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:50.437810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7193bb-9943-4944-8fbc-7ceabd963340 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:50.438576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7193bb-9943-4944-8fbc-7ceabd963340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:50.440269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7193bb-9943-4944-8fbc-7ceabd963340 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:50.452441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 959/1922] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:50.458229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3866d-43ad-4992-a2f8-2c9ec000d55e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:50.481711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3866d-43ad-4992-a2f8-2c9ec000d55e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:50.487747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f12815-cbc5-4e60-a0a6-c1323fbb455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:50.488661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12815-cbc5-4e60-a0a6-c1323fbb455c admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:50.489157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12815-cbc5-4e60-a0a6-c1323fbb455c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:50.489643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12815-cbc5-4e60-a0a6-c1323fbb455c admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:50.490623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f12815-cbc5-4e60-a0a6-c1323fbb455c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:50.492497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f12815-cbc5-4e60-a0a6-c1323fbb455c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:50.510120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 964/1923] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:50.514042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7193bb-9943-4944-8fbc-7ceabd963340 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:50.528171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7193bb-9943-4944-8fbc-7ceabd963340 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:50.533554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694944d-3e14-473a-bc61-1eebd691999e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:50.534445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694944d-3e14-473a-bc61-1eebd691999e admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:50.534733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694944d-3e14-473a-bc61-1eebd691999e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:50.535011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694944d-3e14-473a-bc61-1eebd691999e admin admin] PATH_INFO: `/v3/projects/04ac8653c2464195841b3a4f25eb3b06/users/2aefc8ad70564bfe8b6b221ffce5a0d5/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:50.550534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694944d-3e14-473a-bc61-1eebd691999e admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ac8653c2464195841b3a4f25eb3b06, user_id=2aefc8ad70564bfe8b6b221ffce5a0d5, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:50.553375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694944d-3e14-473a-bc61-1eebd691999e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:50.645902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 960/1924] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:50 2022] PUT /identity/v3/projects/04ac8653c2464195841b3a4f25eb3b06/users/2aefc8ad70564bfe8b6b221ffce5a0d5/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:50.656788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded69560-b9c4-44af-b183-d88bba31cbc8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:50.657086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded69560-b9c4-44af-b183-d88bba31cbc8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:50.657371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded69560-b9c4-44af-b183-d88bba31cbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:50.685415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ded69560-b9c4-44af-b183-d88bba31cbc8 None None] MFA Rules not processed for user `2aefc8ad70564bfe8b6b221ffce5a0d5`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:50.686938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ded69560-b9c4-44af-b183-d88bba31cbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:50.754507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 965/1925] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:50.759017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694944d-3e14-473a-bc61-1eebd691999e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:50.828537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694944d-3e14-473a-bc61-1eebd691999e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:50.836663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405fa66-3b03-47d1-8975-7d38673816c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:50.836663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405fa66-3b03-47d1-8975-7d38673816c0 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:50.837149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405fa66-3b03-47d1-8975-7d38673816c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:50.837544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405fa66-3b03-47d1-8975-7d38673816c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:50.844021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405fa66-3b03-47d1-8975-7d38673816c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:50.844021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405fa66-3b03-47d1-8975-7d38673816c0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:50.846250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:51.018958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 961/1926] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:50 2022] POST /identity/v3/projects => generated 379 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:54:51.023070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded69560-b9c4-44af-b183-d88bba31cbc8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:51.033883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded69560-b9c4-44af-b183-d88bba31cbc8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:51.103185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7abf8-a12e-4e55-8a16-e341666a4285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:51.103899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7abf8-a12e-4e55-8a16-e341666a4285 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.104212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7abf8-a12e-4e55-8a16-e341666a4285 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.104487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7abf8-a12e-4e55-8a16-e341666a4285 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.105174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7abf8-a12e-4e55-8a16-e341666a4285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:51.107160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7abf8-a12e-4e55-8a16-e341666a4285 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:51.235382 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:51.241618 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 966/1927] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:51 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:51.245740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405fa66-3b03-47d1-8975-7d38673816c0 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:51.256648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405fa66-3b03-47d1-8975-7d38673816c0 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:51.263593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326fe44-cdf4-4db1-9ea4-dbc4b49248cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:51.264651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326fe44-cdf4-4db1-9ea4-dbc4b49248cf admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.264998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326fe44-cdf4-4db1-9ea4-dbc4b49248cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.265289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326fe44-cdf4-4db1-9ea4-dbc4b49248cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.266024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326fe44-cdf4-4db1-9ea4-dbc4b49248cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:51.267773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326fe44-cdf4-4db1-9ea4-dbc4b49248cf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:51.280061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 962/1928] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:51.284459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7abf8-a12e-4e55-8a16-e341666a4285 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:51.295771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7abf8-a12e-4e55-8a16-e341666a4285 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:51.300456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7e079-0cc0-4d9f-8beb-75317b04f975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:51.301372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7e079-0cc0-4d9f-8beb-75317b04f975 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.301690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7e079-0cc0-4d9f-8beb-75317b04f975 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.301996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7e079-0cc0-4d9f-8beb-75317b04f975 admin admin] PATH_INFO: `/v3/projects/2b6eb06293cd4e56aeaaef10a7fb8233/users/78419956332f46ee922c82ec693b438f/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.319286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7e079-0cc0-4d9f-8beb-75317b04f975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6eb06293cd4e56aeaaef10a7fb8233, user_id=78419956332f46ee922c82ec693b438f, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:51.322253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7e079-0cc0-4d9f-8beb-75317b04f975 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:51.400790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 967/1929] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:51 2022] PUT /identity/v3/projects/2b6eb06293cd4e56aeaaef10a7fb8233/users/78419956332f46ee922c82ec693b438f/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:51.405709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326fe44-cdf4-4db1-9ea4-dbc4b49248cf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:51.416080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326fe44-cdf4-4db1-9ea4-dbc4b49248cf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:51.488728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba967a-0bf6-46b8-ade9-5a4df906e82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:51.489610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba967a-0bf6-46b8-ade9-5a4df906e82f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.489979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba967a-0bf6-46b8-ade9-5a4df906e82f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.490334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba967a-0bf6-46b8-ade9-5a4df906e82f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.491150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba967a-0bf6-46b8-ade9-5a4df906e82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:51.492964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba967a-0bf6-46b8-ade9-5a4df906e82f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:51.504797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 963/1930] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:51.508858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7e079-0cc0-4d9f-8beb-75317b04f975 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:51.519733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7e079-0cc0-4d9f-8beb-75317b04f975 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:51.524739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:51.525786 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.526146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.526522 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] PATH_INFO: `/v3/projects/2b6eb06293cd4e56aeaaef10a7fb8233/users/78419956332f46ee922c82ec693b438f/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.528809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6eb06293cd4e56aeaaef10a7fb8233, user_id=78419956332f46ee922c82ec693b438f, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:51.531816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:51.558298 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 968/1931] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:51 2022] PUT /identity/v3/projects/2b6eb06293cd4e56aeaaef10a7fb8233/users/78419956332f46ee922c82ec693b438f/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:51.560654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dd278-14f2-44a8-b30e-7ae6f75cf140 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.561502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dd278-14f2-44a8-b30e-7ae6f75cf140 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.561787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dd278-14f2-44a8-b30e-7ae6f75cf140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.589657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-150dd278-14f2-44a8-b30e-7ae6f75cf140 None None] MFA Rules not processed for user `78419956332f46ee922c82ec693b438f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:51.591285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-150dd278-14f2-44a8-b30e-7ae6f75cf140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:51.663711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 964/1932] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:51.675791 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:51.677413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:51.744306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cb954-da0c-4ccb-b9d8-f4a161b0444a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:51.749974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:51.750645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.750932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.751760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.763764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:51.765212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:51.774913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 969/1933] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:51.898801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dd278-14f2-44a8-b30e-7ae6f75cf140 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:51.900336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-150dd278-14f2-44a8-b30e-7ae6f75cf140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:51.916889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dd278-14f2-44a8-b30e-7ae6f75cf140 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:51.921162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdfcd0-61ce-40bb-a1d0-0d49a3a431a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:51.924232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdfcd0-61ce-40bb-a1d0-0d49a3a431a6 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.924232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdfcd0-61ce-40bb-a1d0-0d49a3a431a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.924232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdfcd0-61ce-40bb-a1d0-0d49a3a431a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.982118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:51.985531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bdfcd0-61ce-40bb-a1d0-0d49a3a431a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:51.987005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bdfcd0-61ce-40bb-a1d0-0d49a3a431a6 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:51.997671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 965/1934] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:53.500893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:53.503192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:53.595013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c2bdf-9180-4468-883b-9a0bbd61b7af service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:53.599847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:53.600767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:53.601013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:53.601291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] PATH_INFO: `/v3/users/50e4f561035e4f739c322dd8dbf745bc` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:53.602482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=50e4f561035e4f739c322dd8dbf745bc)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:53.604463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:53.816567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50e4f561035e4f739c322dd8dbf745bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:53.838470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50e4f561035e4f739c322dd8dbf745bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:53.840151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50e4f561035e4f739c322dd8dbf745bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:53.841093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50e4f561035e4f739c322dd8dbf745bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:53.843086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:53.850541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50e4f561035e4f739c322dd8dbf745bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:53.932123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 970/1935] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:53 2022] DELETE /identity/v3/users/50e4f561035e4f739c322dd8dbf745bc => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:53.933758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a369e3-f69d-4e37-a724-ab0d5a6fa326 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:53.934241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a369e3-f69d-4e37-a724-ab0d5a6fa326 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:53.934336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a369e3-f69d-4e37-a724-ab0d5a6fa326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:53.958398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-91a369e3-f69d-4e37-a724-ab0d5a6fa326 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:53.959783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-91a369e3-f69d-4e37-a724-ab0d5a6fa326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:54.028942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 966/1936] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:54.038957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:54.040461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:54.100340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0ec45-22bc-4217-b39b-89a664554acf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:54.103932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74112486-ce43-4ca6-867f-b5fe749ff5dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:54.104575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112486-ce43-4ca6-867f-b5fe749ff5dd service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:54.104860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112486-ce43-4ca6-867f-b5fe749ff5dd service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:54.105140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112486-ce43-4ca6-867f-b5fe749ff5dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:54.114117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74112486-ce43-4ca6-867f-b5fe749ff5dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:54.115574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74112486-ce43-4ca6-867f-b5fe749ff5dd service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:54.125544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 971/1937] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:54.537595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a369e3-f69d-4e37-a724-ab0d5a6fa326 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:54.539064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-91a369e3-f69d-4e37-a724-ab0d5a6fa326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:54.554436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a369e3-f69d-4e37-a724-ab0d5a6fa326 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:54.560285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:54.561737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:54.562014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:54.562290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] PATH_INFO: `/v3/projects/6df5a6ce22e54ad7ae77ea8ca1b03bd7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:54.563579 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df5a6ce22e54ad7ae77ea8ca1b03bd7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:54.565594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:54.689481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df5a6ce22e54ad7ae77ea8ca1b03bd7', 'request_id': 'req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:54.695926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df5a6ce22e54ad7ae77ea8ca1b03bd7', 'request_id': 'req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:54.701701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df5a6ce22e54ad7ae77ea8ca1b03bd7', 'request_id': 'req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:54.776080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] The token cache is being invalidate because project 6df5a6ce22e54ad7ae77ea8ca1b03bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:54.776377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:54.776998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-51d7f9a6-a8b3-44d1-bec2-3d55a43024a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:54.778559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 967/1938] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:54 2022] DELETE /identity/v3/projects/6df5a6ce22e54ad7ae77ea8ca1b03bd7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:54.783432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e83b07-8a2d-4695-bd72-65bb0ba8b275 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:54.783878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e83b07-8a2d-4695-bd72-65bb0ba8b275 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:54.784021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e83b07-8a2d-4695-bd72-65bb0ba8b275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:54.801733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-40e83b07-8a2d-4695-bd72-65bb0ba8b275 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:54.809429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-40e83b07-8a2d-4695-bd72-65bb0ba8b275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:54.862981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 972/1939] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:54.870989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a879da2-7677-472d-aca4-1d06ab5f29c4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:54.874458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a879da2-7677-472d-aca4-1d06ab5f29c4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:54.874458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a879da2-7677-472d-aca4-1d06ab5f29c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:54.891903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9a879da2-7677-472d-aca4-1d06ab5f29c4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:54.893127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9a879da2-7677-472d-aca4-1d06ab5f29c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:54.901054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 968/1940] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:54.905833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e83b07-8a2d-4695-bd72-65bb0ba8b275 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:54.915854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e83b07-8a2d-4695-bd72-65bb0ba8b275 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:54.921041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:54.921700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:54.922153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:54.922440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:54.923164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:54:54.923509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:54.925118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:54.937227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 973/1941] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:54:54.952782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a879da2-7677-472d-aca4-1d06ab5f29c4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:54.952782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a879da2-7677-472d-aca4-1d06ab5f29c4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:54.956831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d56ddb-047c-4b01-b3af-e8a738889b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:54.957615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d56ddb-047c-4b01-b3af-e8a738889b5e admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:54.957824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d56ddb-047c-4b01-b3af-e8a738889b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:54.958105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d56ddb-047c-4b01-b3af-e8a738889b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:54.961603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d56ddb-047c-4b01-b3af-e8a738889b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:54.961603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d56ddb-047c-4b01-b3af-e8a738889b5e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:55.093646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 969/1942] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:54:54 2022] POST /identity/v3/projects => generated 391 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:55.098473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:55.110033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fadaec-88d7-4db9-9b24-bf4954c72fbd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:55.170787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afae57-a9d1-4e03-8db2-8fae59aa416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:55.171516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afae57-a9d1-4e03-8db2-8fae59aa416b admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:55.171808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afae57-a9d1-4e03-8db2-8fae59aa416b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:55.172089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afae57-a9d1-4e03-8db2-8fae59aa416b admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:55.172774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afae57-a9d1-4e03-8db2-8fae59aa416b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:55.174701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afae57-a9d1-4e03-8db2-8fae59aa416b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:55.268793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 974/1943] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:54:55 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:54:55.272837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d56ddb-047c-4b01-b3af-e8a738889b5e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:55.283857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d56ddb-047c-4b01-b3af-e8a738889b5e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:55.288636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ae0e7-98ce-43da-8672-332b2085d0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:55.289365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ae0e7-98ce-43da-8672-332b2085d0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:55.289688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ae0e7-98ce-43da-8672-332b2085d0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:55.289935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ae0e7-98ce-43da-8672-332b2085d0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:55.290709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ae0e7-98ce-43da-8672-332b2085d0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:55.292366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ae0e7-98ce-43da-8672-332b2085d0f8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:55.303156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 970/1944] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:55.307181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afae57-a9d1-4e03-8db2-8fae59aa416b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:55.313165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:55.318691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afae57-a9d1-4e03-8db2-8fae59aa416b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:55.323619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22922e-2680-4ad9-a248-5e764ca88b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:55.324286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22922e-2680-4ad9-a248-5e764ca88b6e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:55.324584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22922e-2680-4ad9-a248-5e764ca88b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:55.327567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22922e-2680-4ad9-a248-5e764ca88b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:55.327567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22922e-2680-4ad9-a248-5e764ca88b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:55.327823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22922e-2680-4ad9-a248-5e764ca88b6e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:55.338581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 975/1945] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:54:55.343795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ae0e7-98ce-43da-8672-332b2085d0f8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:55.353825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ae0e7-98ce-43da-8672-332b2085d0f8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:55.358494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:55.359406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:55.359698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:55.359979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] PATH_INFO: `/v3/projects/fcffedf5fd6e416d935d60b7a0743573/users/89fe54b70d8542fa8939c170cefac7e7/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:55.374525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcffedf5fd6e416d935d60b7a0743573, user_id=89fe54b70d8542fa8939c170cefac7e7, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:55.380986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:55.463480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 971/1946] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:54:55 2022] PUT /identity/v3/projects/fcffedf5fd6e416d935d60b7a0743573/users/89fe54b70d8542fa8939c170cefac7e7/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:55.476793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91915b2-137e-414e-8f63-ff6c66262b4b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:55.477091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91915b2-137e-414e-8f63-ff6c66262b4b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:55.477375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91915b2-137e-414e-8f63-ff6c66262b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:55.502958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d91915b2-137e-414e-8f63-ff6c66262b4b None None] MFA Rules not processed for user `89fe54b70d8542fa8939c170cefac7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:55.504219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d91915b2-137e-414e-8f63-ff6c66262b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:55.569348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 976/1947] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:55.580415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:55.582188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:55.606134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:55.649592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3efdc-10ce-4338-807f-eeb8c94abee7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:55.653315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa272932-e92e-46cf-bf59-af13982ce3fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:55.654082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa272932-e92e-46cf-bf59-af13982ce3fa service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:55.654928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa272932-e92e-46cf-bf59-af13982ce3fa service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:55.655215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa272932-e92e-46cf-bf59-af13982ce3fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:55.663592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa272932-e92e-46cf-bf59-af13982ce3fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:55.665035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa272932-e92e-46cf-bf59-af13982ce3fa service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:55.673793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 972/1948] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:57.636435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91915b2-137e-414e-8f63-ff6c66262b4b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:57.637878 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d91915b2-137e-414e-8f63-ff6c66262b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:57.705315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91915b2-137e-414e-8f63-ff6c66262b4b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:57.709884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:57.710758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:57.711118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:57.711439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] PATH_INFO: `/v3/users/0a8068be6f8549d99ec8f329633905e8` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:57.712735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a8068be6f8549d99ec8f329633905e8)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:57.714743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:57.922845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a8068be6f8549d99ec8f329633905e8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:57.936308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a8068be6f8549d99ec8f329633905e8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:57.937034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a8068be6f8549d99ec8f329633905e8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:57.937641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a8068be6f8549d99ec8f329633905e8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:57.944646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a8068be6f8549d99ec8f329633905e8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:58.015659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:58.020549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 977/1949] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:57 2022] DELETE /identity/v3/users/0a8068be6f8549d99ec8f329633905e8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:58.025350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d649f4-2959-4ac6-a2ac-294dbf0e5ebd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:58.025819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d649f4-2959-4ac6-a2ac-294dbf0e5ebd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:58.025937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d649f4-2959-4ac6-a2ac-294dbf0e5ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:58.047169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-36d649f4-2959-4ac6-a2ac-294dbf0e5ebd None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:54:58.048734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-36d649f4-2959-4ac6-a2ac-294dbf0e5ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:58.111997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 973/1950] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:54:58.123318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:58.124813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:58.189912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935811c3-4c07-4210-b212-12229b1361f9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:58.193678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59313e-0991-4813-b8c7-e301b7a67f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:58.194350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59313e-0991-4813-b8c7-e301b7a67f10 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:58.194637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59313e-0991-4813-b8c7-e301b7a67f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:58.194921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59313e-0991-4813-b8c7-e301b7a67f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:58.206484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59313e-0991-4813-b8c7-e301b7a67f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:58.207979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59313e-0991-4813-b8c7-e301b7a67f10 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:58.218980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 978/1951] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:58.732964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d649f4-2959-4ac6-a2ac-294dbf0e5ebd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:58.734866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-36d649f4-2959-4ac6-a2ac-294dbf0e5ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:58.756427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d649f4-2959-4ac6-a2ac-294dbf0e5ebd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:58.770125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:58.771166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:58.771495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:58.771779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] PATH_INFO: `/v3/projects/272a6838b51946adb8138c31ad203324` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:58.775918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=272a6838b51946adb8138c31ad203324)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:58.778628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:58.899867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '272a6838b51946adb8138c31ad203324', 'request_id': 'req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:58.910434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '272a6838b51946adb8138c31ad203324', 'request_id': 'req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:58.915224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '272a6838b51946adb8138c31ad203324', 'request_id': 'req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.009672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] The token cache is being invalidate because project 272a6838b51946adb8138c31ad203324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:54:59.009899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.010552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.012177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 974/1952] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:54:58 2022] DELETE /identity/v3/projects/272a6838b51946adb8138c31ad203324 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:54:59.016745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59313e-0991-4813-b8c7-e301b7a67f10 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:59.027458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59313e-0991-4813-b8c7-e301b7a67f10 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:59.125577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:59.126577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:59.126957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:59.127550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] PATH_INFO: `/v3/users/2aefc8ad70564bfe8b6b221ffce5a0d5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:59.128714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aefc8ad70564bfe8b6b221ffce5a0d5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:59.130972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:59.421603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aefc8ad70564bfe8b6b221ffce5a0d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.442595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aefc8ad70564bfe8b6b221ffce5a0d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.442595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aefc8ad70564bfe8b6b221ffce5a0d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.442595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aefc8ad70564bfe8b6b221ffce5a0d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.449727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aefc8ad70564bfe8b6b221ffce5a0d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.547588 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 979/1953] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:59 2022] DELETE /identity/v3/users/2aefc8ad70564bfe8b6b221ffce5a0d5 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:54:59.640942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:59.642448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:59.655532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d0ab7-8fd1-40cc-89a8-b7216b3954bf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:59.698352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:59.720387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:59.720387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:59.720387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:59.720387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] PATH_INFO: `/v3/users/28c12b17379f4a4baf5d8b24f1aecbc8` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:59.720825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c12b17379f4a4baf5d8b24f1aecbc8)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:59.721680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:59.973093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c12b17379f4a4baf5d8b24f1aecbc8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.986120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c12b17379f4a4baf5d8b24f1aecbc8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.986810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c12b17379f4a4baf5d8b24f1aecbc8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.989041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c12b17379f4a4baf5d8b24f1aecbc8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:54:59.996512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d8a26e89-2b3a-4f9f-b276-68de1744b760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c12b17379f4a4baf5d8b24f1aecbc8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.023379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:00.024891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:00.039553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa8d44-de0e-469c-b9cf-2b20fac554d4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:00.044571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:00.045505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:00.045759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:00.046008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] PATH_INFO: `/v3/projects/04ac8653c2464195841b3a4f25eb3b06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:00.047200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ac8653c2464195841b3a4f25eb3b06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:00.049171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:00.070390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:00.099562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 975/1954] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:54:59 2022] DELETE /identity/v3/users/28c12b17379f4a4baf5d8b24f1aecbc8 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:00.104054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23680c-1384-432f-89a1-ac1b32b993bc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:00.104356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23680c-1384-432f-89a1-ac1b32b993bc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:00.104754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23680c-1384-432f-89a1-ac1b32b993bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:00.121913 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-aa23680c-1384-432f-89a1-ac1b32b993bc None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:00.123643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa23680c-1384-432f-89a1-ac1b32b993bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:00.193129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 976/1955] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:00.203510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23680c-1384-432f-89a1-ac1b32b993bc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:00.204944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa23680c-1384-432f-89a1-ac1b32b993bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:00.211633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ac8653c2464195841b3a4f25eb3b06', 'request_id': 'req-2dfff53f-3107-467f-b5be-9235cb09fa52'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.217830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ac8653c2464195841b3a4f25eb3b06', 'request_id': 'req-2dfff53f-3107-467f-b5be-9235cb09fa52'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.262289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23680c-1384-432f-89a1-ac1b32b993bc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:00.274843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:00.290330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ac8653c2464195841b3a4f25eb3b06', 'request_id': 'req-2dfff53f-3107-467f-b5be-9235cb09fa52'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.302413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] The token cache is being invalidate because project 04ac8653c2464195841b3a4f25eb3b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:00.302987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.304578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.307158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 980/1956] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:00 2022] DELETE /identity/v3/projects/04ac8653c2464195841b3a4f25eb3b06 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:00.312135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:00.314266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:00.319718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:00.320743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:00.321222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:00.321681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:00.324268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:00.397824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff53f-3107-467f-b5be-9235cb09fa52 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:00.402489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:00.405115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:00.405237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:00.405237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:00.405237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] PATH_INFO: `/v3/users/78419956332f46ee922c82ec693b438f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:00.405441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:00.405513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78419956332f46ee922c82ec693b438f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:00.407157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:00.414623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 977/1957] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:00.647278 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78419956332f46ee922c82ec693b438f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.665751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78419956332f46ee922c82ec693b438f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.666509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78419956332f46ee922c82ec693b438f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.667193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78419956332f46ee922c82ec693b438f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.675276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78419956332f46ee922c82ec693b438f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:00.760086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 981/1958] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:00 2022] DELETE /identity/v3/users/78419956332f46ee922c82ec693b438f => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:00.899236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:00.900827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:00.913953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e8ccc-ea7d-40d0-8555-d45c218b7f8b service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:00.985157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:00.986060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:00.986702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:00.986702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] PATH_INFO: `/v3/projects/1111d08f11fa40189407b3d0ce6df136` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:00.987853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1111d08f11fa40189407b3d0ce6df136)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:00.989816 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:01.089334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1111d08f11fa40189407b3d0ce6df136', 'request_id': 'req-0bcce520-58a1-44e4-a8d4-96846311835c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.097230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1111d08f11fa40189407b3d0ce6df136', 'request_id': 'req-0bcce520-58a1-44e4-a8d4-96846311835c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.118200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1111d08f11fa40189407b3d0ce6df136', 'request_id': 'req-0bcce520-58a1-44e4-a8d4-96846311835c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.262756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:01.270509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] The token cache is being invalidate because project 1111d08f11fa40189407b3d0ce6df136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:01.270509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.270509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.271084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 978/1959] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:00 2022] DELETE /identity/v3/projects/1111d08f11fa40189407b3d0ce6df136 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:01.275215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:01.276730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:01.290751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c6bce-8de1-450b-9e1a-26025908a740 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:01.312109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:01.344072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:01.345618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:01.350806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:01.352014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.352313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.352647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] PATH_INFO: `/v3/users/6e1caab2e6be4fac96de16bc499bb727` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.353892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1caab2e6be4fac96de16bc499bb727)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:01.357071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:01.360714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcce520-58a1-44e4-a8d4-96846311835c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:01.369695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:01.370632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.370894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.371217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] PATH_INFO: `/v3/projects/2b6eb06293cd4e56aeaaef10a7fb8233` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.372945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b6eb06293cd4e56aeaaef10a7fb8233)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:01.375279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:01.633085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1caab2e6be4fac96de16bc499bb727'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.633966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b6eb06293cd4e56aeaaef10a7fb8233', 'request_id': 'req-21c35d28-e5a8-4b47-8458-9be8e9dc213b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.641814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b6eb06293cd4e56aeaaef10a7fb8233', 'request_id': 'req-21c35d28-e5a8-4b47-8458-9be8e9dc213b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.647679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b6eb06293cd4e56aeaaef10a7fb8233', 'request_id': 'req-21c35d28-e5a8-4b47-8458-9be8e9dc213b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.648999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1caab2e6be4fac96de16bc499bb727'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.650486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1caab2e6be4fac96de16bc499bb727'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.651987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1caab2e6be4fac96de16bc499bb727'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.659976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1caab2e6be4fac96de16bc499bb727'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.773487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] The token cache is being invalidate because project 2b6eb06293cd4e56aeaaef10a7fb8233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:01.773789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.780071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-21c35d28-e5a8-4b47-8458-9be8e9dc213b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.780071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 979/1960] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:01 2022] DELETE /identity/v3/projects/2b6eb06293cd4e56aeaaef10a7fb8233 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:01.784352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817d1e5-72dc-457b-91fc-9d275e76e4b7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.784650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817d1e5-72dc-457b-91fc-9d275e76e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.784939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817d1e5-72dc-457b-91fc-9d275e76e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.804289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6817d1e5-72dc-457b-91fc-9d275e76e4b7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:01.805535 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6817d1e5-72dc-457b-91fc-9d275e76e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:01.863708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 980/1961] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:01.872719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d89be-82e3-42d2-97fd-2e248dce48bb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.873022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d89be-82e3-42d2-97fd-2e248dce48bb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.873302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d89be-82e3-42d2-97fd-2e248dce48bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.891730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3f0d89be-82e3-42d2-97fd-2e248dce48bb None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:01.892949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3f0d89be-82e3-42d2-97fd-2e248dce48bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:01.902344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 981/1962] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:01.907011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d89be-82e3-42d2-97fd-2e248dce48bb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:01.916411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d89be-82e3-42d2-97fd-2e248dce48bb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:01.917227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 982/1963] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:01 2022] DELETE /identity/v3/users/6e1caab2e6be4fac96de16bc499bb727 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:01.973627 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:01.974311 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.974587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.986502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.986502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:01.986502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:01.986502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:01.993940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 982/1964] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:55:01.995393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:01.996876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:02.014792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87f623-dae9-43ee-a67e-9f0f31356cb5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:02.022219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08945f-4ff5-42b0-84e1-ef765a6aa4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:02.023400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08945f-4ff5-42b0-84e1-ef765a6aa4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.023915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08945f-4ff5-42b0-84e1-ef765a6aa4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.024419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08945f-4ff5-42b0-84e1-ef765a6aa4b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.025726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08945f-4ff5-42b0-84e1-ef765a6aa4b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:02.030058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08945f-4ff5-42b0-84e1-ef765a6aa4b2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:02.236672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 983/1965] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:01 2022] POST /identity/v3/projects => generated 359 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:02.240478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:02.250736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa9426-7e69-4b32-9fd1-e57341cefc79 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:02.255551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:02.310064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384103ed-8de6-4061-a461-60627f17bb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:02.310785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384103ed-8de6-4061-a461-60627f17bb43 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.311190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384103ed-8de6-4061-a461-60627f17bb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.311520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384103ed-8de6-4061-a461-60627f17bb43 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.312243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384103ed-8de6-4061-a461-60627f17bb43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:02.314211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384103ed-8de6-4061-a461-60627f17bb43 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:02.403278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 983/1966] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:02 2022] POST /identity/v3/users => generated 280 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:02.411070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08945f-4ff5-42b0-84e1-ef765a6aa4b2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:02.421691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08945f-4ff5-42b0-84e1-ef765a6aa4b2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:02.426553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d6a98-551e-4350-9b6e-85146cdf0e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:02.427227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d6a98-551e-4350-9b6e-85146cdf0e64 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.427570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d6a98-551e-4350-9b6e-85146cdf0e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.430944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d6a98-551e-4350-9b6e-85146cdf0e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.430944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d6a98-551e-4350-9b6e-85146cdf0e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:02.430944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d6a98-551e-4350-9b6e-85146cdf0e64 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:02.441458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 984/1967] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:55:02.445576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384103ed-8de6-4061-a461-60627f17bb43 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:02.455568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384103ed-8de6-4061-a461-60627f17bb43 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:02.460241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:02.460895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.461180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.461461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.462177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:02.463830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:02.475094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 984/1968] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:02.484283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d6a98-551e-4350-9b6e-85146cdf0e64 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:02.494373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d6a98-551e-4350-9b6e-85146cdf0e64 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:02.498954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605d69a-bc1f-4be1-a192-13ddd3077675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:02.499874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605d69a-bc1f-4be1-a192-13ddd3077675 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.500170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605d69a-bc1f-4be1-a192-13ddd3077675 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.500434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605d69a-bc1f-4be1-a192-13ddd3077675 admin admin] PATH_INFO: `/v3/projects/eb96f28950994546ae417828fbf8c3b6/users/b3586157cfd64eeb97b422ff05d3bc7f/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.507336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:02.509051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:02.526800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73275d96-869a-4465-9724-6bceee2f7efa admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:02.526943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605d69a-bc1f-4be1-a192-13ddd3077675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb96f28950994546ae417828fbf8c3b6, user_id=b3586157cfd64eeb97b422ff05d3bc7f, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:02.530848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605d69a-bc1f-4be1-a192-13ddd3077675 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:02.532717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:02.533920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.534430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.534885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] PATH_INFO: `/v3/projects/28bafc988eda41adbb5956605cd81472` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.536336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bafc988eda41adbb5956605cd81472)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:02.538314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:02.657009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 985/1969] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:02 2022] PUT /identity/v3/projects/eb96f28950994546ae417828fbf8c3b6/users/b3586157cfd64eeb97b422ff05d3bc7f/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:02.669252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148290d-98e9-4bda-bc96-c696cb986b93 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.669547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148290d-98e9-4bda-bc96-c696cb986b93 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.670385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148290d-98e9-4bda-bc96-c696cb986b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.696682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6148290d-98e9-4bda-bc96-c696cb986b93 None None] MFA Rules not processed for user `b3586157cfd64eeb97b422ff05d3bc7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:02.697912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6148290d-98e9-4bda-bc96-c696cb986b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:02.731404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:02.754088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bafc988eda41adbb5956605cd81472', 'request_id': 'req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.760217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bafc988eda41adbb5956605cd81472', 'request_id': 'req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.762067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 986/1970] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:02 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:02.766489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bafc988eda41adbb5956605cd81472', 'request_id': 'req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.794468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148290d-98e9-4bda-bc96-c696cb986b93 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:02.798979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6148290d-98e9-4bda-bc96-c696cb986b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:02.853256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148290d-98e9-4bda-bc96-c696cb986b93 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:02.868517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] The token cache is being invalidate because project 28bafc988eda41adbb5956605cd81472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:02.869167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.869982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1e6f7a47-e5b8-4f94-ae9e-51a20d63f614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.877203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 985/1971] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:02 2022] DELETE /identity/v3/projects/28bafc988eda41adbb5956605cd81472 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:02.887532 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc63fc-38cf-4cc9-bf91-b8839902a706 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.888129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc63fc-38cf-4cc9-bf91-b8839902a706 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.888968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc63fc-38cf-4cc9-bf91-b8839902a706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.906066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4f4f0-8efb-4e0c-9abc-eee70cc10ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:02.906970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4f4f0-8efb-4e0c-9abc-eee70cc10ebb service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.911421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4f4f0-8efb-4e0c-9abc-eee70cc10ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.911421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4f4f0-8efb-4e0c-9abc-eee70cc10ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.916544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9db4f4f0-8efb-4e0c-9abc-eee70cc10ebb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:02.923203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ebbc63fc-38cf-4cc9-bf91-b8839902a706 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:02.924456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ebbc63fc-38cf-4cc9-bf91-b8839902a706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:02.989929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4f4f0-8efb-4e0c-9abc-eee70cc10ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:02.992210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4f4f0-8efb-4e0c-9abc-eee70cc10ebb service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.003014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 987/1972] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:02 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:03.008352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 986/1973] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:03.016433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d631a0-3ee9-42bf-9b0c-70b483686d12 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.016837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d631a0-3ee9-42bf-9b0c-70b483686d12 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.017125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d631a0-3ee9-42bf-9b0c-70b483686d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.017386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc63fc-38cf-4cc9-bf91-b8839902a706 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.018898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ebbc63fc-38cf-4cc9-bf91-b8839902a706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:03.031332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc63fc-38cf-4cc9-bf91-b8839902a706 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.037176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a5d631a0-3ee9-42bf-9b0c-70b483686d12 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:03.037176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a5d631a0-3ee9-42bf-9b0c-70b483686d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:03.037367 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.037494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.037761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.038043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] PATH_INFO: `/v3/users/b3586157cfd64eeb97b422ff05d3bc7f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.039253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3586157cfd64eeb97b422ff05d3bc7f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.041214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.045239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 988/1974] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:03.049447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d631a0-3ee9-42bf-9b0c-70b483686d12 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.058315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d631a0-3ee9-42bf-9b0c-70b483686d12 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.062987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.063802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.064090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.064373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.065089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:03.065398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.067009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.080428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 989/1975] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:55:03.082394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:03.084781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.095189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d3490-18aa-494e-a865-025b3e9be380 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.099800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a650615-c395-4779-ac13-373e569011d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.100431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a650615-c395-4779-ac13-373e569011d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.100717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a650615-c395-4779-ac13-373e569011d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.100993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a650615-c395-4779-ac13-373e569011d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.101668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a650615-c395-4779-ac13-373e569011d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.103579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a650615-c395-4779-ac13-373e569011d0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.245433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 990/1976] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:03 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:03.250980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a650615-c395-4779-ac13-373e569011d0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.267743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3586157cfd64eeb97b422ff05d3bc7f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:03.279688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3586157cfd64eeb97b422ff05d3bc7f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:03.280416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3586157cfd64eeb97b422ff05d3bc7f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:03.281024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3586157cfd64eeb97b422ff05d3bc7f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:03.288440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f0d62895-520a-4d05-a42d-1880a1180c77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3586157cfd64eeb97b422ff05d3bc7f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:03.327959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a650615-c395-4779-ac13-373e569011d0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.332783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.333461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.333747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.334104 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.335280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.340666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.380072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 987/1977] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:03 2022] DELETE /identity/v3/users/b3586157cfd64eeb97b422ff05d3bc7f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:03.384837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c1ff-4003-4e26-9cc9-d50884a868e1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.385137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c1ff-4003-4e26-9cc9-d50884a868e1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.385422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539c1ff-4003-4e26-9cc9-d50884a868e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.403268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2539c1ff-4003-4e26-9cc9-d50884a868e1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:03.407056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2539c1ff-4003-4e26-9cc9-d50884a868e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:03.444779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:03.454464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 991/1978] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:03 2022] POST /identity/v3/users => generated 283 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:03.458822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.460677 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:03.472259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 988/1979] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:03.483379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ccb11-76b7-4350-879a-021d94990554 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.483379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5d38f-3434-4953-9ac8-a0926501bafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.483635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539c1ff-4003-4e26-9cc9-d50884a868e1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.483755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d38f-3434-4953-9ac8-a0926501bafb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.483755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d38f-3434-4953-9ac8-a0926501bafb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.483755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5d38f-3434-4953-9ac8-a0926501bafb admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.483755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5d38f-3434-4953-9ac8-a0926501bafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.484066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2539c1ff-4003-4e26-9cc9-d50884a868e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:03.484826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff5d38f-3434-4953-9ac8-a0926501bafb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.495215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 992/1980] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:03.500058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5d38f-3434-4953-9ac8-a0926501bafb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.509989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff5d38f-3434-4953-9ac8-a0926501bafb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.515489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf40cb6-c643-4bc3-8f19-a563421e3120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.516150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40cb6-c643-4bc3-8f19-a563421e3120 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.516438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40cb6-c643-4bc3-8f19-a563421e3120 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.516863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf40cb6-c643-4bc3-8f19-a563421e3120 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.517561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf40cb6-c643-4bc3-8f19-a563421e3120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.519304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf40cb6-c643-4bc3-8f19-a563421e3120 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.530203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 993/1981] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:55:03.534619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf40cb6-c643-4bc3-8f19-a563421e3120 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.546259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf40cb6-c643-4bc3-8f19-a563421e3120 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.549872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539c1ff-4003-4e26-9cc9-d50884a868e1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.551969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2445923-8827-4814-9a77-3699491b625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.552987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445923-8827-4814-9a77-3699491b625f admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.553346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445923-8827-4814-9a77-3699491b625f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.553693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445923-8827-4814-9a77-3699491b625f admin admin] PATH_INFO: `/v3/projects/ecfd10cad4d543b9995b7184d19d6bfc/users/35e2f28c583e4f5da5ecf4632189ebf9/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.554387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7f534-7994-4e1c-af66-62f10058a9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.555214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f534-7994-4e1c-af66-62f10058a9da service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.555541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f534-7994-4e1c-af66-62f10058a9da service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.556286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7f534-7994-4e1c-af66-62f10058a9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.566717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7f534-7994-4e1c-af66-62f10058a9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.568269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7f534-7994-4e1c-af66-62f10058a9da service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.573182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2445923-8827-4814-9a77-3699491b625f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecfd10cad4d543b9995b7184d19d6bfc, user_id=35e2f28c583e4f5da5ecf4632189ebf9, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.576833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2445923-8827-4814-9a77-3699491b625f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:03.578561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 989/1982] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:03.723106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 994/1983] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:03 2022] PUT /identity/v3/projects/ecfd10cad4d543b9995b7184d19d6bfc/users/35e2f28c583e4f5da5ecf4632189ebf9/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:03.733482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a8f23-b44a-4726-877e-e8ee25caa494 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.733778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a8f23-b44a-4726-877e-e8ee25caa494 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.734059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a8f23-b44a-4726-877e-e8ee25caa494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.760580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-850a8f23-b44a-4726-877e-e8ee25caa494 None None] MFA Rules not processed for user `35e2f28c583e4f5da5ecf4632189ebf9`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:03.761836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-850a8f23-b44a-4726-877e-e8ee25caa494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:03.775712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:03.831806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 990/1984] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:03.836270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2445923-8827-4814-9a77-3699491b625f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:03.901233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2445923-8827-4814-9a77-3699491b625f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:03.906920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3562792-f2b9-4757-9073-5344e3461ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:03.907620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3562792-f2b9-4757-9073-5344e3461ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:03.907912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3562792-f2b9-4757-9073-5344e3461ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:03.908189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3562792-f2b9-4757-9073-5344e3461ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:03.908893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3562792-f2b9-4757-9073-5344e3461ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:03.910817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3562792-f2b9-4757-9073-5344e3461ee0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.029344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a8f23-b44a-4726-877e-e8ee25caa494 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.032865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-850a8f23-b44a-4726-877e-e8ee25caa494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:04.054755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a8f23-b44a-4726-877e-e8ee25caa494 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.060779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.061808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.062235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.062727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] PATH_INFO: `/v3/projects/eb96f28950994546ae417828fbf8c3b6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.064152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb96f28950994546ae417828fbf8c3b6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.065675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 995/1985] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:03 2022] POST /identity/v3/projects => generated 363 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:04.068531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.069406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3562792-f2b9-4757-9073-5344e3461ee0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.115940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:04.144154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3562792-f2b9-4757-9073-5344e3461ee0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.149509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2195ca-ee16-4de9-9103-4925325f62e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.150224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2195ca-ee16-4de9-9103-4925325f62e3 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.155712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2195ca-ee16-4de9-9103-4925325f62e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.155712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2195ca-ee16-4de9-9103-4925325f62e3 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.155712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2195ca-ee16-4de9-9103-4925325f62e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.155712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2195ca-ee16-4de9-9103-4925325f62e3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.201994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb96f28950994546ae417828fbf8c3b6', 'request_id': 'req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:04.208135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb96f28950994546ae417828fbf8c3b6', 'request_id': 'req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:04.214260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb96f28950994546ae417828fbf8c3b6', 'request_id': 'req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:04.271039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 996/1986] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:04 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:04.278187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2195ca-ee16-4de9-9103-4925325f62e3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.288717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2195ca-ee16-4de9-9103-4925325f62e3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.301051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] The token cache is being invalidate because project eb96f28950994546ae417828fbf8c3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:04.301382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:04.302013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dd496a1-770c-4d3c-9bfa-13f31092a5ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:04.304487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 991/1987] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:04 2022] DELETE /identity/v3/projects/eb96f28950994546ae417828fbf8c3b6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:04.308634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490ce0-9721-40ba-9e93-d76bef84300c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.308930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490ce0-9721-40ba-9e93-d76bef84300c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.309214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490ce0-9721-40ba-9e93-d76bef84300c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.327941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-98490ce0-9721-40ba-9e93-d76bef84300c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:04.333059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-98490ce0-9721-40ba-9e93-d76bef84300c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:04.348992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.350187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.350474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.350756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.351534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.353172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.363629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 997/1988] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:55:04.369007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.369481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:04.382964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef328b-273f-4094-b23e-c9e6bad7145a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.388864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dad48-289f-463d-8ca2-cb8e1b4010e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.389932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dad48-289f-463d-8ca2-cb8e1b4010e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.390218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dad48-289f-463d-8ca2-cb8e1b4010e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.390958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dad48-289f-463d-8ca2-cb8e1b4010e6 admin admin] PATH_INFO: `/v3/projects/b755115674ab4882983ec02b533758e6/users/d4b23218d3a046f9b0a4b7fe389327b3/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.407906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dad48-289f-463d-8ca2-cb8e1b4010e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b755115674ab4882983ec02b533758e6, user_id=d4b23218d3a046f9b0a4b7fe389327b3, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.410966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dad48-289f-463d-8ca2-cb8e1b4010e6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.411437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 992/1989] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:04.416379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c6f4-6a85-4251-b8df-3c7b92b5415f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.416682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c6f4-6a85-4251-b8df-3c7b92b5415f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.416971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796c6f4-6a85-4251-b8df-3c7b92b5415f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.435503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e796c6f4-6a85-4251-b8df-3c7b92b5415f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:04.436777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e796c6f4-6a85-4251-b8df-3c7b92b5415f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:04.444823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 993/1990] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:04.448928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796c6f4-6a85-4251-b8df-3c7b92b5415f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.458472 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796c6f4-6a85-4251-b8df-3c7b92b5415f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.463132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.463792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.464123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.464405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.465422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:04.469024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.469024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.479488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 994/1991] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:55:04.483486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.492993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f6ad-fe06-43ff-a61d-ae3712342c1f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.493733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 998/1992] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:04 2022] PUT /identity/v3/projects/b755115674ab4882983ec02b533758e6/users/d4b23218d3a046f9b0a4b7fe389327b3/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:04.500158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dad48-289f-463d-8ca2-cb8e1b4010e6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.511425 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:04.551681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329fa50-fc29-4ea9-9149-e5943459d88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.552323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329fa50-fc29-4ea9-9149-e5943459d88a admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.552616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329fa50-fc29-4ea9-9149-e5943459d88a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.552897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329fa50-fc29-4ea9-9149-e5943459d88a admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.553587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329fa50-fc29-4ea9-9149-e5943459d88a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.555588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329fa50-fc29-4ea9-9149-e5943459d88a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.562983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dad48-289f-463d-8ca2-cb8e1b4010e6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.567782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6071a-92b2-476f-8404-59e0b9deddea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.568450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6071a-92b2-476f-8404-59e0b9deddea admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.568737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6071a-92b2-476f-8404-59e0b9deddea admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.569012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6071a-92b2-476f-8404-59e0b9deddea admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.569733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6071a-92b2-476f-8404-59e0b9deddea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.571367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6071a-92b2-476f-8404-59e0b9deddea admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.581939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 999/1993] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:55:04.586607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6071a-92b2-476f-8404-59e0b9deddea admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.597293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6071a-92b2-476f-8404-59e0b9deddea admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.601956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec26c2-64e6-421d-ab1f-a1003ec1adae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.602832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec26c2-64e6-421d-ab1f-a1003ec1adae admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.603114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec26c2-64e6-421d-ab1f-a1003ec1adae admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.603422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec26c2-64e6-421d-ab1f-a1003ec1adae admin admin] PATH_INFO: `/v3/projects/b755115674ab4882983ec02b533758e6/users/d4b23218d3a046f9b0a4b7fe389327b3/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.605390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec26c2-64e6-421d-ab1f-a1003ec1adae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b755115674ab4882983ec02b533758e6, user_id=d4b23218d3a046f9b0a4b7fe389327b3, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.608050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec26c2-64e6-421d-ab1f-a1003ec1adae admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.618688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1000/1994] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:04 2022] PUT /identity/v3/projects/b755115674ab4882983ec02b533758e6/users/d4b23218d3a046f9b0a4b7fe389327b3/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:04.630896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268bc21-9fc8-4c9c-9752-936228dff620 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.631181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268bc21-9fc8-4c9c-9752-936228dff620 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.635329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268bc21-9fc8-4c9c-9752-936228dff620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.641795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 995/1995] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:04 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:04.645866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329fa50-fc29-4ea9-9149-e5943459d88a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.664997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9268bc21-9fc8-4c9c-9752-936228dff620 None None] MFA Rules not processed for user `d4b23218d3a046f9b0a4b7fe389327b3`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:04.668479 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9268bc21-9fc8-4c9c-9752-936228dff620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:04.674796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:04.725925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329fa50-fc29-4ea9-9149-e5943459d88a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.731558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bc70b-5105-4609-bbfd-51d594dfca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.732288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc70b-5105-4609-bbfd-51d594dfca1f admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.732579 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc70b-5105-4609-bbfd-51d594dfca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.733085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2bc70b-5105-4609-bbfd-51d594dfca1f admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.733684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bc70b-5105-4609-bbfd-51d594dfca1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.735667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2bc70b-5105-4609-bbfd-51d594dfca1f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.753259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1001/1996] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:04 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:04.765093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268bc21-9fc8-4c9c-9752-936228dff620 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.766493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9268bc21-9fc8-4c9c-9752-936228dff620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:04.823626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268bc21-9fc8-4c9c-9752-936228dff620 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.827869 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3728c-ae2c-4a3c-baab-cebdca49b775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.828400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3728c-ae2c-4a3c-baab-cebdca49b775 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.828912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 996/1997] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:04 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:04.829096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3728c-ae2c-4a3c-baab-cebdca49b775 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.829096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3728c-ae2c-4a3c-baab-cebdca49b775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.832722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bc70b-5105-4609-bbfd-51d594dfca1f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.842179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3728c-ae2c-4a3c-baab-cebdca49b775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.843725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3728c-ae2c-4a3c-baab-cebdca49b775 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.845132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2bc70b-5105-4609-bbfd-51d594dfca1f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.850414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d3be-0660-4936-978b-b1bff5204a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.851245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d3be-0660-4936-978b-b1bff5204a24 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.851676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d3be-0660-4936-978b-b1bff5204a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.852050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d3be-0660-4936-978b-b1bff5204a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.853265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0d3be-0660-4936-978b-b1bff5204a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.854263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1002/1998] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:04 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:04.854712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0d3be-0660-4936-978b-b1bff5204a24 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.868301 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 997/1999] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:55:04.872346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3728c-ae2c-4a3c-baab-cebdca49b775 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.882948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3728c-ae2c-4a3c-baab-cebdca49b775 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.887671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5d94c-6e68-4ad8-b8bc-d90c06d26fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.888355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5d94c-6e68-4ad8-b8bc-d90c06d26fcb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.888638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5d94c-6e68-4ad8-b8bc-d90c06d26fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.888913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5d94c-6e68-4ad8-b8bc-d90c06d26fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.889622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5d94c-6e68-4ad8-b8bc-d90c06d26fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.891204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5d94c-6e68-4ad8-b8bc-d90c06d26fcb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:04.901443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1003/2000] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:04.908507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d3be-0660-4936-978b-b1bff5204a24 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:04.918633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d3be-0660-4936-978b-b1bff5204a24 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:04.923890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047aefc-3f8c-4c8f-aaa8-83deb0c81e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:04.924768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047aefc-3f8c-4c8f-aaa8-83deb0c81e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:04.925049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047aefc-3f8c-4c8f-aaa8-83deb0c81e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:04.925326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047aefc-3f8c-4c8f-aaa8-83deb0c81e41 admin admin] PATH_INFO: `/v3/projects/13d3fe5e587640d1a817f6583470e418/users/50172204540545c992c46b13e1bd168e/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:04.940903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047aefc-3f8c-4c8f-aaa8-83deb0c81e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d3fe5e587640d1a817f6583470e418, user_id=50172204540545c992c46b13e1bd168e, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:04.943674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047aefc-3f8c-4c8f-aaa8-83deb0c81e41 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:05.034440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 998/2001] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:04 2022] PUT /identity/v3/projects/13d3fe5e587640d1a817f6583470e418/users/50172204540545c992c46b13e1bd168e/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:05.045002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c1fa-f79e-455a-8663-9db064a55b2c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.045452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c1fa-f79e-455a-8663-9db064a55b2c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.045743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5c1fa-f79e-455a-8663-9db064a55b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:05.074725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9cb5c1fa-f79e-455a-8663-9db064a55b2c None None] MFA Rules not processed for user `50172204540545c992c46b13e1bd168e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:05.076790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9cb5c1fa-f79e-455a-8663-9db064a55b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:05.121374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047aefc-3f8c-4c8f-aaa8-83deb0c81e41 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:05.146545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1004/2002] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:05.150538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5c1fa-f79e-455a-8663-9db064a55b2c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:05.177636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:05.179102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047aefc-3f8c-4c8f-aaa8-83deb0c81e41 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:05.183274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:05.183966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.184274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.184553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:05.187755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:05.203285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:05.229646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5c1fa-f79e-455a-8663-9db064a55b2c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:05.236000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b86ae-9233-4853-befa-7a055276f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:05.237528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b86ae-9233-4853-befa-7a055276f0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.238122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b86ae-9233-4853-befa-7a055276f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.238584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b86ae-9233-4853-befa-7a055276f0f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:05.239838 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b86ae-9233-4853-befa-7a055276f0f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:05.242923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b86ae-9233-4853-befa-7a055276f0f1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:05.255955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:05.258555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:05.271269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 999/2003] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:05.400083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1005/2004] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:05 2022] POST /identity/v3/projects => generated 373 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:05.404228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:05.415188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b716a1-2797-4354-b26d-a3c27badbfb2 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:05.471884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8319e3-722b-4583-a376-4d58fda872c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:05.472580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8319e3-722b-4583-a376-4d58fda872c7 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.472867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8319e3-722b-4583-a376-4d58fda872c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.473149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8319e3-722b-4583-a376-4d58fda872c7 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:05.473844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8319e3-722b-4583-a376-4d58fda872c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:05.475821 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8319e3-722b-4583-a376-4d58fda872c7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:05.597330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1000/2005] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:05 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:05.601258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b86ae-9233-4853-befa-7a055276f0f1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:05.611701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b86ae-9233-4853-befa-7a055276f0f1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:05.616472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc15e21-dab3-4028-b96d-592e44a50364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:05.617132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc15e21-dab3-4028-b96d-592e44a50364 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.617416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc15e21-dab3-4028-b96d-592e44a50364 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.617694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc15e21-dab3-4028-b96d-592e44a50364 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:05.618558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc15e21-dab3-4028-b96d-592e44a50364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:05.620196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc15e21-dab3-4028-b96d-592e44a50364 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:05.631637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1006/2006] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:05.635684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8319e3-722b-4583-a376-4d58fda872c7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:05.646651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8319e3-722b-4583-a376-4d58fda872c7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:05.651204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da7ca3-a617-4c58-a88c-982a33582cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:05.652159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da7ca3-a617-4c58-a88c-982a33582cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.652449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da7ca3-a617-4c58-a88c-982a33582cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.652738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da7ca3-a617-4c58-a88c-982a33582cfe admin admin] PATH_INFO: `/v3/projects/2a4ea84e0261470daff80fc72de42825/users/0bc4d210d4764acca0966fb53677ed81/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:05.667598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da7ca3-a617-4c58-a88c-982a33582cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4ea84e0261470daff80fc72de42825, user_id=0bc4d210d4764acca0966fb53677ed81, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:05.670314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da7ca3-a617-4c58-a88c-982a33582cfe admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:05.813671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1001/2007] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:05 2022] PUT /identity/v3/projects/2a4ea84e0261470daff80fc72de42825/users/0bc4d210d4764acca0966fb53677ed81/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:05.817915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc15e21-dab3-4028-b96d-592e44a50364 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:05.829822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc15e21-dab3-4028-b96d-592e44a50364 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:05.898989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e883d46-6c58-4b57-8ef0-a7f57343aead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:05.900076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e883d46-6c58-4b57-8ef0-a7f57343aead admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.900646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e883d46-6c58-4b57-8ef0-a7f57343aead admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.901023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e883d46-6c58-4b57-8ef0-a7f57343aead admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:05.902020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e883d46-6c58-4b57-8ef0-a7f57343aead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:05.904386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e883d46-6c58-4b57-8ef0-a7f57343aead admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:05.914874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1007/2008] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:05.919030 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da7ca3-a617-4c58-a88c-982a33582cfe admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:05.930748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da7ca3-a617-4c58-a88c-982a33582cfe admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:05.937318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5b4a-f584-495a-a255-744afd7c65e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:05.938218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5b4a-f584-495a-a255-744afd7c65e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.938507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5b4a-f584-495a-a255-744afd7c65e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.938793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5b4a-f584-495a-a255-744afd7c65e5 admin admin] PATH_INFO: `/v3/projects/2a4ea84e0261470daff80fc72de42825/users/0bc4d210d4764acca0966fb53677ed81/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:05.942893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac5b4a-f584-495a-a255-744afd7c65e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4ea84e0261470daff80fc72de42825, user_id=0bc4d210d4764acca0966fb53677ed81, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:05.945636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac5b4a-f584-495a-a255-744afd7c65e5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:05.957862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1002/2009] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:05 2022] PUT /identity/v3/projects/2a4ea84e0261470daff80fc72de42825/users/0bc4d210d4764acca0966fb53677ed81/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:05.969991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fc5e2-84fc-42a7-8789-1044806a6c17 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:05.970345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fc5e2-84fc-42a7-8789-1044806a6c17 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:05.970689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fc5e2-84fc-42a7-8789-1044806a6c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:06.003605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0f6fc5e2-84fc-42a7-8789-1044806a6c17 None None] MFA Rules not processed for user `0bc4d210d4764acca0966fb53677ed81`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:06.004875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0f6fc5e2-84fc-42a7-8789-1044806a6c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:06.080404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1008/2010] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:06.092097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5b4a-f584-495a-a255-744afd7c65e5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:06.152313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5b4a-f584-495a-a255-744afd7c65e5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:06.156761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f47584-b1ed-4c77-8008-67f774ad90b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:06.157415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47584-b1ed-4c77-8008-67f774ad90b4 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:06.157708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47584-b1ed-4c77-8008-67f774ad90b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:06.157996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f47584-b1ed-4c77-8008-67f774ad90b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:06.177975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:06.215411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f47584-b1ed-4c77-8008-67f774ad90b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:06.217431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f47584-b1ed-4c77-8008-67f774ad90b4 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:06.226260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1003/2011] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:16.077288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:16.507248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=18 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:26.214826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fc5e2-84fc-42a7-8789-1044806a6c17 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:26.290106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fc5e2-84fc-42a7-8789-1044806a6c17 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:26.295802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:26.296683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:26.296975 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:26.297252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] PATH_INFO: `/v3/users/50172204540545c992c46b13e1bd168e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:26.298447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50172204540545c992c46b13e1bd168e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:26.300457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:26.571724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50172204540545c992c46b13e1bd168e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:26.584978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50172204540545c992c46b13e1bd168e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:26.585685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50172204540545c992c46b13e1bd168e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:26.586288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50172204540545c992c46b13e1bd168e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:26.594244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50172204540545c992c46b13e1bd168e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:26.682571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1009/2012] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:26 2022] DELETE /identity/v3/users/50172204540545c992c46b13e1bd168e => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:26.687104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96db5b6-1b96-4cb7-89fc-ba132a67b81d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:26.687877 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96db5b6-1b96-4cb7-89fc-ba132a67b81d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:26.688056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96db5b6-1b96-4cb7-89fc-ba132a67b81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:26.705691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f96db5b6-1b96-4cb7-89fc-ba132a67b81d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:26.706907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f96db5b6-1b96-4cb7-89fc-ba132a67b81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:26.768620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1004/2013] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:26.787379 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:26.788807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:26.845627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a31845-9a8c-4a36-9786-3cdbf6f30211 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:26.849447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1decd6-5eb3-40b8-af19-b1e8092a8df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:26.850098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1decd6-5eb3-40b8-af19-b1e8092a8df1 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:26.850387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1decd6-5eb3-40b8-af19-b1e8092a8df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:26.850673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1decd6-5eb3-40b8-af19-b1e8092a8df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:26.859973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1decd6-5eb3-40b8-af19-b1e8092a8df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:26.861443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1decd6-5eb3-40b8-af19-b1e8092a8df1 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:26.880345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1010/2014] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:27.336751 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96db5b6-1b96-4cb7-89fc-ba132a67b81d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:27.338921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f96db5b6-1b96-4cb7-89fc-ba132a67b81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:27.365419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96db5b6-1b96-4cb7-89fc-ba132a67b81d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:27.374145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:27.375594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:27.375594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:27.375925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] PATH_INFO: `/v3/projects/13d3fe5e587640d1a817f6583470e418` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:27.377239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d3fe5e587640d1a817f6583470e418)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:27.384245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:27.527159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d3fe5e587640d1a817f6583470e418', 'request_id': 'req-1de02d76-2293-4e95-8181-1d6301b09cdb'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:27.537018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d3fe5e587640d1a817f6583470e418', 'request_id': 'req-1de02d76-2293-4e95-8181-1d6301b09cdb'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:27.549942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d3fe5e587640d1a817f6583470e418', 'request_id': 'req-1de02d76-2293-4e95-8181-1d6301b09cdb'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:27.621405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] The token cache is being invalidate because project 13d3fe5e587640d1a817f6583470e418 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:27.627275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:27.627275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:27.627275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1005/2015] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:27 2022] DELETE /identity/v3/projects/13d3fe5e587640d1a817f6583470e418 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:27.628784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1decd6-5eb3-40b8-af19-b1e8092a8df1 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:27.639284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1decd6-5eb3-40b8-af19-b1e8092a8df1 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:27.664211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:27.723641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:27.727405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:27.727405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:27.727405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] PATH_INFO: `/v3/users/0bc4d210d4764acca0966fb53677ed81` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:27.728487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc4d210d4764acca0966fb53677ed81)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:27.732329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:27.991708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc4d210d4764acca0966fb53677ed81'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:28.012316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc4d210d4764acca0966fb53677ed81'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:28.013069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc4d210d4764acca0966fb53677ed81'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:28.013685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc4d210d4764acca0966fb53677ed81'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:28.020625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5ea3623c-3e04-464b-98a5-95ddece5b462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc4d210d4764acca0966fb53677ed81'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:28.149489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1011/2016] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:27 2022] DELETE /identity/v3/users/0bc4d210d4764acca0966fb53677ed81 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:28.734046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:28.735580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:28.752971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de02d76-2293-4e95-8181-1d6301b09cdb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:28.808762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:28.809651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:28.809939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:28.810266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] PATH_INFO: `/v3/projects/2a4ea84e0261470daff80fc72de42825` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:28.811735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4ea84e0261470daff80fc72de42825)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:28.813700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:28.941245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:28.958177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4ea84e0261470daff80fc72de42825', 'request_id': 'req-1449793b-a0ed-4771-beee-bcf29318ac88'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:28.964802 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4ea84e0261470daff80fc72de42825', 'request_id': 'req-1449793b-a0ed-4771-beee-bcf29318ac88'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:28.971133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4ea84e0261470daff80fc72de42825', 'request_id': 'req-1449793b-a0ed-4771-beee-bcf29318ac88'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:29.127880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] The token cache is being invalidate because project 2a4ea84e0261470daff80fc72de42825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:29.128088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:29.128725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1449793b-a0ed-4771-beee-bcf29318ac88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:29.130309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1006/2017] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:28 2022] DELETE /identity/v3/projects/2a4ea84e0261470daff80fc72de42825 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:29.135081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45afab-3bbf-4825-9d6b-eca0979163d3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.135569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45afab-3bbf-4825-9d6b-eca0979163d3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.135819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45afab-3bbf-4825-9d6b-eca0979163d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.155812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7b45afab-3bbf-4825-9d6b-eca0979163d3 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:29.161705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7b45afab-3bbf-4825-9d6b-eca0979163d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:29.216747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1012/2018] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:29.224375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554a00-8a8f-4cc8-93c7-4f369e373a9f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.224668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554a00-8a8f-4cc8-93c7-4f369e373a9f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.224949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554a00-8a8f-4cc8-93c7-4f369e373a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.243168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8a554a00-8a8f-4cc8-93c7-4f369e373a9f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:29.244407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8a554a00-8a8f-4cc8-93c7-4f369e373a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:29.252499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1007/2019] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:29.256746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45afab-3bbf-4825-9d6b-eca0979163d3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:29.270009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45afab-3bbf-4825-9d6b-eca0979163d3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:29.272416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:29.273076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.273387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.273652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.274378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:29.274688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:29.276327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:29.289360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1013/2020] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:55:29.293902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a554a00-8a8f-4cc8-93c7-4f369e373a9f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:29.303058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a554a00-8a8f-4cc8-93c7-4f369e373a9f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:29.308054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ac497-7a8e-4704-832b-3beb3f9bfcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:29.308705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ac497-7a8e-4704-832b-3beb3f9bfcaa admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.309021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ac497-7a8e-4704-832b-3beb3f9bfcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.309298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ac497-7a8e-4704-832b-3beb3f9bfcaa admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.309994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ac497-7a8e-4704-832b-3beb3f9bfcaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:29.311945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ac497-7a8e-4704-832b-3beb3f9bfcaa admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:29.444950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1008/2021] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:29 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:29.449123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:29.460726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef962907-2bca-4e8d-a016-eded01c88c78 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:29.489630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:29.520834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e34040-d617-40cd-a139-43c7442b61b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:29.521404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e34040-d617-40cd-a139-43c7442b61b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.521672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e34040-d617-40cd-a139-43c7442b61b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.521956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e34040-d617-40cd-a139-43c7442b61b6 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.522770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e34040-d617-40cd-a139-43c7442b61b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:29.524728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e34040-d617-40cd-a139-43c7442b61b6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:29.640648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1014/2022] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:29 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:29.645208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ac497-7a8e-4704-832b-3beb3f9bfcaa admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:29.658215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ac497-7a8e-4704-832b-3beb3f9bfcaa admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:29.663476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e020e-ddbe-4ce8-8e56-681480c3de4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:29.664161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e020e-ddbe-4ce8-8e56-681480c3de4c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.664447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e020e-ddbe-4ce8-8e56-681480c3de4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.669069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e020e-ddbe-4ce8-8e56-681480c3de4c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.669962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e020e-ddbe-4ce8-8e56-681480c3de4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:29.671660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e020e-ddbe-4ce8-8e56-681480c3de4c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:29.685413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1009/2023] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:29.686911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e34040-d617-40cd-a139-43c7442b61b6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:29.698985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e34040-d617-40cd-a139-43c7442b61b6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:29.705486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05733e41-7f42-4603-9e2e-177e96f87601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:29.710413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05733e41-7f42-4603-9e2e-177e96f87601 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.710413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05733e41-7f42-4603-9e2e-177e96f87601 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.710413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05733e41-7f42-4603-9e2e-177e96f87601 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.710413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05733e41-7f42-4603-9e2e-177e96f87601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:29.710413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05733e41-7f42-4603-9e2e-177e96f87601 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:29.724625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1015/2024] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:29.725671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e020e-ddbe-4ce8-8e56-681480c3de4c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:29.737287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e020e-ddbe-4ce8-8e56-681480c3de4c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:29.743062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c9154-8e47-47fa-a145-57c154b4152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:29.744889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c9154-8e47-47fa-a145-57c154b4152d admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.744889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c9154-8e47-47fa-a145-57c154b4152d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.744889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c9154-8e47-47fa-a145-57c154b4152d admin admin] PATH_INFO: `/v3/projects/5031fbad9e18426484f5fffcd9c01bc4/users/ba6099c1f81a42c89ccbc33e07f9e26a/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.761203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c9154-8e47-47fa-a145-57c154b4152d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5031fbad9e18426484f5fffcd9c01bc4, user_id=ba6099c1f81a42c89ccbc33e07f9e26a, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:29.764435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c9154-8e47-47fa-a145-57c154b4152d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:29.884792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1010/2025] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:29 2022] PUT /identity/v3/projects/5031fbad9e18426484f5fffcd9c01bc4/users/ba6099c1f81a42c89ccbc33e07f9e26a/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:29.895606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ceddd-c5e7-4772-81af-e5dde4376fa9 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:29.895902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ceddd-c5e7-4772-81af-e5dde4376fa9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:29.896177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ceddd-c5e7-4772-81af-e5dde4376fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:29.936169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5b5ceddd-c5e7-4772-81af-e5dde4376fa9 None None] MFA Rules not processed for user `ba6099c1f81a42c89ccbc33e07f9e26a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:29.937463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5b5ceddd-c5e7-4772-81af-e5dde4376fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:30.036271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1016/2026] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:30.056151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c9154-8e47-47fa-a145-57c154b4152d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.135849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c9154-8e47-47fa-a145-57c154b4152d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:30.140852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:30.141527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.141945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.142236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.142932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:30.144908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:30.238990 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1011/2027] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:30 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:30.243272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ceddd-c5e7-4772-81af-e5dde4376fa9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.254123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ceddd-c5e7-4772-81af-e5dde4376fa9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:30.310327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c559ab-6267-4c1c-8bed-694207689760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:30.310923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c559ab-6267-4c1c-8bed-694207689760 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.311248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c559ab-6267-4c1c-8bed-694207689760 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.311556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c559ab-6267-4c1c-8bed-694207689760 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.312241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c559ab-6267-4c1c-8bed-694207689760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:30.314167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c559ab-6267-4c1c-8bed-694207689760 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:30.458658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.464922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:30.482833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d8438-2ecc-4de5-975c-1f984ce45f1d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:30.490499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:30.490853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.491250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.491577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] PATH_INFO: `/v3/users/89fe54b70d8542fa8939c170cefac7e7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.492838 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] RBAC: Authorizing `identity:delete_user(user_id=89fe54b70d8542fa8939c170cefac7e7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:30.495339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1017/2028] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:30 2022] POST /identity/v3/users => generated 289 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:30.495466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:30.499485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c559ab-6267-4c1c-8bed-694207689760 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.523749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c559ab-6267-4c1c-8bed-694207689760 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:30.530172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3fc1a-8e33-481e-afcf-c0dad03a8569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:30.530911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3fc1a-8e33-481e-afcf-c0dad03a8569 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.531137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3fc1a-8e33-481e-afcf-c0dad03a8569 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.531462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3fc1a-8e33-481e-afcf-c0dad03a8569 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.532201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3fc1a-8e33-481e-afcf-c0dad03a8569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:30.533840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3fc1a-8e33-481e-afcf-c0dad03a8569 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:30.544913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1018/2029] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:55:30.548972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3fc1a-8e33-481e-afcf-c0dad03a8569 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.558882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3fc1a-8e33-481e-afcf-c0dad03a8569 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:30.564271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a688-7015-4999-a08d-d7a1c52f69dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:30.565022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a688-7015-4999-a08d-d7a1c52f69dd admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.565381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a688-7015-4999-a08d-d7a1c52f69dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.565669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20a688-7015-4999-a08d-d7a1c52f69dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.566432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a688-7015-4999-a08d-d7a1c52f69dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:30.568154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20a688-7015-4999-a08d-d7a1c52f69dd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:30.583380 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1019/2030] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:55:30.584463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a688-7015-4999-a08d-d7a1c52f69dd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.592201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:30.595947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20a688-7015-4999-a08d-d7a1c52f69dd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:30.600754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8461dd2-694b-411f-8ec4-92e63e47937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:30.603400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8461dd2-694b-411f-8ec4-92e63e47937d admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.603400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8461dd2-694b-411f-8ec4-92e63e47937d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.603400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8461dd2-694b-411f-8ec4-92e63e47937d admin admin] PATH_INFO: `/v3/projects/8a1fe94ee8744e6da47af5c719a6b413/users/e238bfa601eb4470b99399288984b629/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.616330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8461dd2-694b-411f-8ec4-92e63e47937d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1fe94ee8744e6da47af5c719a6b413, user_id=e238bfa601eb4470b99399288984b629, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:30.618977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8461dd2-694b-411f-8ec4-92e63e47937d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:30.686525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1020/2031] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:30 2022] PUT /identity/v3/projects/8a1fe94ee8744e6da47af5c719a6b413/users/e238bfa601eb4470b99399288984b629/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:30.699447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6701d-c99b-4e78-9685-1f9552707240 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.700236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6701d-c99b-4e78-9685-1f9552707240 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.700521 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6701d-c99b-4e78-9685-1f9552707240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.728883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8dd6701d-c99b-4e78-9685-1f9552707240 None None] MFA Rules not processed for user `e238bfa601eb4470b99399288984b629`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:30.730142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8dd6701d-c99b-4e78-9685-1f9552707240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:30.741001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:30.754977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89fe54b70d8542fa8939c170cefac7e7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:30.767426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89fe54b70d8542fa8939c170cefac7e7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:30.768134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89fe54b70d8542fa8939c170cefac7e7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:30.768729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89fe54b70d8542fa8939c170cefac7e7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:30.775418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e28853a5-d1b8-4070-a5d9-8d177166631e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89fe54b70d8542fa8939c170cefac7e7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:30.795489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1021/2032] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:30.800052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6701d-c99b-4e78-9685-1f9552707240 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.801385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8dd6701d-c99b-4e78-9685-1f9552707240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:30.866824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6701d-c99b-4e78-9685-1f9552707240 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:30.872194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:30.872898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.873191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.873474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.874167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:30.876128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:30.885405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1012/2033] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:30 2022] DELETE /identity/v3/users/89fe54b70d8542fa8939c170cefac7e7 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:30.890386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264c313-31aa-4295-b96c-8092a81efce4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:30.890871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264c313-31aa-4295-b96c-8092a81efce4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:30.891153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264c313-31aa-4295-b96c-8092a81efce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:30.911509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0264c313-31aa-4295-b96c-8092a81efce4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:30.912845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0264c313-31aa-4295-b96c-8092a81efce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:30.972391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1013/2034] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:30.973982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1022/2035] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:30 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:30.978372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264c313-31aa-4295-b96c-8092a81efce4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.981808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:30.983578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:30.988892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264c313-31aa-4295-b96c-8092a81efce4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.051476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6c5ad-a2db-43ad-99cf-5f587f85be81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.052165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6c5ad-a2db-43ad-99cf-5f587f85be81 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.052456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6c5ad-a2db-43ad-99cf-5f587f85be81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.052741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6c5ad-a2db-43ad-99cf-5f587f85be81 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.053434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6c5ad-a2db-43ad-99cf-5f587f85be81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.055586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6c5ad-a2db-43ad-99cf-5f587f85be81 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:31.061095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e22c-356f-4196-9793-7920e37befad admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.065166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6fb65-fe26-4dcf-9c67-f0e8b8b07faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.065829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6fb65-fe26-4dcf-9c67-f0e8b8b07faf service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.066116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6fb65-fe26-4dcf-9c67-f0e8b8b07faf service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.066393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6fb65-fe26-4dcf-9c67-f0e8b8b07faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.076876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6fb65-fe26-4dcf-9c67-f0e8b8b07faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.078346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6fb65-fe26-4dcf-9c67-f0e8b8b07faf service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:31.088317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1023/2036] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:31.198711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1014/2037] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:30 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:31.203432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6fb65-fe26-4dcf-9c67-f0e8b8b07faf service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:31.213455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6fb65-fe26-4dcf-9c67-f0e8b8b07faf service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.220705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:31.221981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9e6e-92e2-41d2-9eba-03f492dcafe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.222740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9e6e-92e2-41d2-9eba-03f492dcafe8 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.223040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9e6e-92e2-41d2-9eba-03f492dcafe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.223360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c9e6e-92e2-41d2-9eba-03f492dcafe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.224150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c9e6e-92e2-41d2-9eba-03f492dcafe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.225954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c9e6e-92e2-41d2-9eba-03f492dcafe8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:31.237347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1024/2038] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:31.241749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6c5ad-a2db-43ad-99cf-5f587f85be81 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:31.252190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6c5ad-a2db-43ad-99cf-5f587f85be81 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.257143 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425cbce-66ea-408e-a461-86ccc4347c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.257893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425cbce-66ea-408e-a461-86ccc4347c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.258181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425cbce-66ea-408e-a461-86ccc4347c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.258486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425cbce-66ea-408e-a461-86ccc4347c32 admin admin] PATH_INFO: `/v3/projects/ff3ff4087b5e4a1eb2036e0b40da3141/users/0d71281db6de4d82ad0b75f51042dca9/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.274105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425cbce-66ea-408e-a461-86ccc4347c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3ff4087b5e4a1eb2036e0b40da3141, user_id=0d71281db6de4d82ad0b75f51042dca9, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.276833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425cbce-66ea-408e-a461-86ccc4347c32 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:31.377924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1015/2039] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:31 2022] PUT /identity/v3/projects/ff3ff4087b5e4a1eb2036e0b40da3141/users/0d71281db6de4d82ad0b75f51042dca9/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:31.381768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9e6e-92e2-41d2-9eba-03f492dcafe8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:31.395538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c9e6e-92e2-41d2-9eba-03f492dcafe8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.456417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b24ca-a21b-494b-b16a-50e32f79b343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.457197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b24ca-a21b-494b-b16a-50e32f79b343 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.457482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b24ca-a21b-494b-b16a-50e32f79b343 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.457777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b24ca-a21b-494b-b16a-50e32f79b343 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.458787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b24ca-a21b-494b-b16a-50e32f79b343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.461843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b24ca-a21b-494b-b16a-50e32f79b343 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:31.474234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1025/2040] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:31.478257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425cbce-66ea-408e-a461-86ccc4347c32 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:31.499004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425cbce-66ea-408e-a461-86ccc4347c32 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.506368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.511736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.512154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.512456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] PATH_INFO: `/v3/projects/ff3ff4087b5e4a1eb2036e0b40da3141/users/0d71281db6de4d82ad0b75f51042dca9/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.515209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff3ff4087b5e4a1eb2036e0b40da3141, user_id=0d71281db6de4d82ad0b75f51042dca9, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.518046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:31.529351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1016/2041] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:31 2022] PUT /identity/v3/projects/ff3ff4087b5e4a1eb2036e0b40da3141/users/0d71281db6de4d82ad0b75f51042dca9/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:31.539500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f35288-c519-4482-bf4b-cc1726005fc9 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.539802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f35288-c519-4482-bf4b-cc1726005fc9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.540087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f35288-c519-4482-bf4b-cc1726005fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.568859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e1f35288-c519-4482-bf4b-cc1726005fc9 None None] MFA Rules not processed for user `0d71281db6de4d82ad0b75f51042dca9`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:31.570082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e1f35288-c519-4482-bf4b-cc1726005fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:31.603518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:31.606957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:31.710207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:31.713497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1026/2042] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:31.730857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f35288-c519-4482-bf4b-cc1726005fc9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:31.732312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e1f35288-c519-4482-bf4b-cc1726005fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:31.733119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c98d7f-8d0a-4e00-8330-b267dd032aec admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.738371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.740287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.740584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.740867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] PATH_INFO: `/v3/projects/fcffedf5fd6e416d935d60b7a0743573` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.742327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcffedf5fd6e416d935d60b7a0743573)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.744333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:31.796030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f35288-c519-4482-bf4b-cc1726005fc9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.799662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.800303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.800591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.800864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.802634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:31.823087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:31.863743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.864253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:31.875108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1027/2043] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:31.877184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:31.878673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcffedf5fd6e416d935d60b7a0743573', 'request_id': 'req-ac9883a4-9f8d-461b-9252-7487adf32593'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:31.878816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:31.885914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcffedf5fd6e416d935d60b7a0743573', 'request_id': 'req-ac9883a4-9f8d-461b-9252-7487adf32593'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:31.904178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcffedf5fd6e416d935d60b7a0743573', 'request_id': 'req-ac9883a4-9f8d-461b-9252-7487adf32593'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:31.904400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a5fe9-c0cf-478c-824c-cf51b9bd6e5b service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:31.955037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:31.955951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:31.956240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:31.956518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] PATH_INFO: `/v3/users/35e2f28c583e4f5da5ecf4632189ebf9` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:31.957741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e2f28c583e4f5da5ecf4632189ebf9)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:31.959743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:32.073072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] The token cache is being invalidate because project fcffedf5fd6e416d935d60b7a0743573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:32.073402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:32.074062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ac9883a4-9f8d-461b-9252-7487adf32593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:32.078744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1017/2044] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:31 2022] DELETE /identity/v3/projects/fcffedf5fd6e416d935d60b7a0743573 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:32.083594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03263257-be46-463f-8f49-eaef63da84cd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.084154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03263257-be46-463f-8f49-eaef63da84cd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.084432 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03263257-be46-463f-8f49-eaef63da84cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.103436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-03263257-be46-463f-8f49-eaef63da84cd None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:32.107412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-03263257-be46-463f-8f49-eaef63da84cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:32.126969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1018/2045] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:32.131495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abac4f-f23c-4e0e-8fc8-e6e853c6da35 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.131819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abac4f-f23c-4e0e-8fc8-e6e853c6da35 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.132105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abac4f-f23c-4e0e-8fc8-e6e853c6da35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.159968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b7abac4f-f23c-4e0e-8fc8-e6e853c6da35 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:32.161279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b7abac4f-f23c-4e0e-8fc8-e6e853c6da35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:32.173542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1019/2046] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:32.176690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abac4f-f23c-4e0e-8fc8-e6e853c6da35 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:32.187809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abac4f-f23c-4e0e-8fc8-e6e853c6da35 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:32.193424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:32.194314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.194705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.195774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.196794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:32.197211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:32.199091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:32.215941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1020/2047] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:55:32.222887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:32.235958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71851c-5ae7-4d03-8beb-51a53cf2c1c1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:32.241621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:32.242293 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.243026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.243026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.243582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:32.245678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:32.303829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e2f28c583e4f5da5ecf4632189ebf9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:32.318307 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e2f28c583e4f5da5ecf4632189ebf9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:32.318653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e2f28c583e4f5da5ecf4632189ebf9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:32.319277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e2f28c583e4f5da5ecf4632189ebf9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:32.334142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3f56ded-2f02-4c9f-b3db-da839652cf95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e2f28c583e4f5da5ecf4632189ebf9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:32.396820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1021/2048] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:32 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:55:32.404028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:32.406059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:32.422638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1028/2049] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:31 2022] DELETE /identity/v3/users/35e2f28c583e4f5da5ecf4632189ebf9 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:32.428758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77abc5c-f488-4cd2-ac04-2dc32443e3ca None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.429071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77abc5c-f488-4cd2-ac04-2dc32443e3ca None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.429353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77abc5c-f488-4cd2-ac04-2dc32443e3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.450020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a77abc5c-f488-4cd2-ac04-2dc32443e3ca None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:32.451654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a77abc5c-f488-4cd2-ac04-2dc32443e3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:32.481835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:32.505165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:32.506785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8d7f7-258a-40cc-859d-7c90dee80b5d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:32.542894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1029/2050] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:32.559461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77abc5c-f488-4cd2-ac04-2dc32443e3ca None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:32.561424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a77abc5c-f488-4cd2-ac04-2dc32443e3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:32.584844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0453b0c-44db-4ef3-8d48-9ef3101ec5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:32.585663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0453b0c-44db-4ef3-8d48-9ef3101ec5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.586079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0453b0c-44db-4ef3-8d48-9ef3101ec5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.586460 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0453b0c-44db-4ef3-8d48-9ef3101ec5f7 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.587262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0453b0c-44db-4ef3-8d48-9ef3101ec5f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:32.589608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0453b0c-44db-4ef3-8d48-9ef3101ec5f7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:32.630934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77abc5c-f488-4cd2-ac04-2dc32443e3ca None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:32.634959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99eeb-5cf1-436e-aa5b-dc6686e4f2b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:32.635727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99eeb-5cf1-436e-aa5b-dc6686e4f2b7 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.636080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99eeb-5cf1-436e-aa5b-dc6686e4f2b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.636426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec99eeb-5cf1-436e-aa5b-dc6686e4f2b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.647139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec99eeb-5cf1-436e-aa5b-dc6686e4f2b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:32.648672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec99eeb-5cf1-436e-aa5b-dc6686e4f2b7 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:32.661900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1030/2051] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:32.710429 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1022/2052] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:32 2022] POST /identity/v3/users => generated 292 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:32.711420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99eeb-5cf1-436e-aa5b-dc6686e4f2b7 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:32.725063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec99eeb-5cf1-436e-aa5b-dc6686e4f2b7 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:32.733950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb186679-7d19-4de5-8e84-be4af3fd07cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:32.733950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb186679-7d19-4de5-8e84-be4af3fd07cb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.733950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb186679-7d19-4de5-8e84-be4af3fd07cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.733950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb186679-7d19-4de5-8e84-be4af3fd07cb admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.734416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb186679-7d19-4de5-8e84-be4af3fd07cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:32.736061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb186679-7d19-4de5-8e84-be4af3fd07cb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:32.752510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1031/2053] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:55:32.756900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0453b0c-44db-4ef3-8d48-9ef3101ec5f7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:32.769298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0453b0c-44db-4ef3-8d48-9ef3101ec5f7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:32.774903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0584eba-e389-4f3f-8916-4a832e789f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:32.775774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0584eba-e389-4f3f-8916-4a832e789f6e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.776123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0584eba-e389-4f3f-8916-4a832e789f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.776474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0584eba-e389-4f3f-8916-4a832e789f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.777381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0584eba-e389-4f3f-8916-4a832e789f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:32.780128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0584eba-e389-4f3f-8916-4a832e789f6e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:32.792271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1023/2054] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:32.797834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb186679-7d19-4de5-8e84-be4af3fd07cb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:32.811593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb186679-7d19-4de5-8e84-be4af3fd07cb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:32.825638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:32.826636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.826976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.828103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] PATH_INFO: `/v3/projects/06d54c4865ff45e59526d477accb6cc4/users/bff41a6af0ea495a856fd5603e810d6a/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.828224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca3dc2-686c-4553-a3e6-a3648a0d024a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.828388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca3dc2-686c-4553-a3e6-a3648a0d024a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.828677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca3dc2-686c-4553-a3e6-a3648a0d024a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:32.844592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d54c4865ff45e59526d477accb6cc4, user_id=bff41a6af0ea495a856fd5603e810d6a, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:32.847594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:32.858478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d0ca3dc2-686c-4553-a3e6-a3648a0d024a None None] MFA Rules not processed for user `a3efd91d94b44c6aa41087c2f017861f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:32.859762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d0ca3dc2-686c-4553-a3e6-a3648a0d024a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:32.924868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1024/2055] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:32.966827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1032/2056] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:32 2022] PUT /identity/v3/projects/06d54c4865ff45e59526d477accb6cc4/users/bff41a6af0ea495a856fd5603e810d6a/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:32.972409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd099a-2b50-4166-903d-d045246b215d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:32.972830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd099a-2b50-4166-903d-d045246b215d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:32.973216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd099a-2b50-4166-903d-d045246b215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:33.007231 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-73dd099a-2b50-4166-903d-d045246b215d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:33.009616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-73dd099a-2b50-4166-903d-d045246b215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:33.087587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1025/2057] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:33.108619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:33.110893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:33.191330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2545ca7-ebd7-4da3-8a61-591d7402b2c0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:33.201053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2a34e-43de-405e-b921-f60b91cad351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:33.202186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2a34e-43de-405e-b921-f60b91cad351 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:33.202665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2a34e-43de-405e-b921-f60b91cad351 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:33.203257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2a34e-43de-405e-b921-f60b91cad351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:33.216182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2a34e-43de-405e-b921-f60b91cad351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:33.218464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2a34e-43de-405e-b921-f60b91cad351 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:33.230329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1033/2058] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:33.373691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd099a-2b50-4166-903d-d045246b215d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:33.375217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-73dd099a-2b50-4166-903d-d045246b215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:33.387824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd099a-2b50-4166-903d-d045246b215d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:33.393031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:33.393609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:33.393948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:33.394226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] PATH_INFO: `/v3/projects/ecfd10cad4d543b9995b7184d19d6bfc` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:33.395482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecfd10cad4d543b9995b7184d19d6bfc)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:33.398125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:33.408489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:33.564683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecfd10cad4d543b9995b7184d19d6bfc', 'request_id': 'req-4acfd674-7ce3-4919-8c91-03042f834d12'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:33.570941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecfd10cad4d543b9995b7184d19d6bfc', 'request_id': 'req-4acfd674-7ce3-4919-8c91-03042f834d12'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:33.576831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecfd10cad4d543b9995b7184d19d6bfc', 'request_id': 'req-4acfd674-7ce3-4919-8c91-03042f834d12'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:33.742154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] The token cache is being invalidate because project ecfd10cad4d543b9995b7184d19d6bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:33.742805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:33.743650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:33.745466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1026/2059] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:33 2022] DELETE /identity/v3/projects/ecfd10cad4d543b9995b7184d19d6bfc => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:33.749636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2a34e-43de-405e-b921-f60b91cad351 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:33.765290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2a34e-43de-405e-b921-f60b91cad351 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:33.865123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:33.885147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:33.886062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:33.886352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:33.886634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] PATH_INFO: `/v3/users/d4b23218d3a046f9b0a4b7fe389327b3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:33.887914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4b23218d3a046f9b0a4b7fe389327b3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:33.890015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:34.146052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4b23218d3a046f9b0a4b7fe389327b3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:34.158255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4b23218d3a046f9b0a4b7fe389327b3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:34.158955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4b23218d3a046f9b0a4b7fe389327b3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:34.159599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4b23218d3a046f9b0a4b7fe389327b3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:34.166636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4b23218d3a046f9b0a4b7fe389327b3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:34.317420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1034/2060] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:33 2022] DELETE /identity/v3/users/d4b23218d3a046f9b0a4b7fe389327b3 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:34.895952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:34.897996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:34.922845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfd674-7ce3-4919-8c91-03042f834d12 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:35.040818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:35.042019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:35.042314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:35.042595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] PATH_INFO: `/v3/projects/b755115674ab4882983ec02b533758e6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:35.045055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b755115674ab4882983ec02b533758e6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:35.047025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:35.304175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b755115674ab4882983ec02b533758e6', 'request_id': 'req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:35.331637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b755115674ab4882983ec02b533758e6', 'request_id': 'req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:35.341742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:35.343187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:35.348702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b755115674ab4882983ec02b533758e6', 'request_id': 'req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:35.426793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf25a0-70a3-49b2-a648-19134e28f6c9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:35.456297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] The token cache is being invalidate because project b755115674ab4882983ec02b533758e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:35.456754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:35.457444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-578504cd-eaa4-4dfc-aec4-d683ab8bbf96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:35.459771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1027/2061] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:34 2022] DELETE /identity/v3/projects/b755115674ab4882983ec02b533758e6 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:35.464712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0bf6-5431-4357-9eeb-2f96c2abb498 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:35.465181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0bf6-5431-4357-9eeb-2f96c2abb498 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:35.465622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0bf6-5431-4357-9eeb-2f96c2abb498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:35.489159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:35.490258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:35.490685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:35.491119 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:35.493005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:35.493201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-032c0bf6-5431-4357-9eeb-2f96c2abb498 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:35.494521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-032c0bf6-5431-4357-9eeb-2f96c2abb498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:35.517863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:35.556223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:35.577359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:35.577799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1028/2062] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:35.579208 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:35.586016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5d176-5a5e-4ed7-bc8b-a1d937330d86 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:35.586442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5d176-5a5e-4ed7-bc8b-a1d937330d86 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:35.586990 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5d176-5a5e-4ed7-bc8b-a1d937330d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:35.593038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1035/2063] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:35.638911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fec5d176-5a5e-4ed7-bc8b-a1d937330d86 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:35.655651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fec5d176-5a5e-4ed7-bc8b-a1d937330d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:35.682056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1029/2064] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:35.687831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:35.709134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627617dc-b853-4e99-86ce-0036e7220529 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:35.724093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:35.729595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:35.729595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:35.729595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:35.729595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:35.729595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:35.729976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:35.749148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1036/2065] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:55:35.751377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5d176-5a5e-4ed7-bc8b-a1d937330d86 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:35.766018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5d176-5a5e-4ed7-bc8b-a1d937330d86 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:35.772461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7507c13-b2b5-4889-961f-885edf837e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:35.773964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7507c13-b2b5-4889-961f-885edf837e4f admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:35.774400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7507c13-b2b5-4889-961f-885edf837e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:35.774788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7507c13-b2b5-4889-961f-885edf837e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:35.775763 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7507c13-b2b5-4889-961f-885edf837e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:35.781278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7507c13-b2b5-4889-961f-885edf837e4f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:35.901955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1030/2066] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:35 2022] POST /identity/v3/projects => generated 369 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:35.907326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:35.939185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c110e-2f75-4c4e-82ff-7e28addc239f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:36.061586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29436954-602d-4554-9d4e-0ad064866792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:36.062304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29436954-602d-4554-9d4e-0ad064866792 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:36.062597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29436954-602d-4554-9d4e-0ad064866792 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:36.062890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29436954-602d-4554-9d4e-0ad064866792 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:36.063636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29436954-602d-4554-9d4e-0ad064866792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:36.066876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29436954-602d-4554-9d4e-0ad064866792 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:36.184743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1037/2067] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:35 2022] POST /identity/v3/users => generated 285 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:36.188787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7507c13-b2b5-4889-961f-885edf837e4f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:36.202735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7507c13-b2b5-4889-961f-885edf837e4f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:36.218653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6740d7-c4ca-4db0-8831-fd92090d9b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:36.219424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6740d7-c4ca-4db0-8831-fd92090d9b18 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:36.219729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6740d7-c4ca-4db0-8831-fd92090d9b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:36.220019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6740d7-c4ca-4db0-8831-fd92090d9b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:36.220770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6740d7-c4ca-4db0-8831-fd92090d9b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:36.224282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6740d7-c4ca-4db0-8831-fd92090d9b18 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:36.250294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1031/2068] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:36.254459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29436954-602d-4554-9d4e-0ad064866792 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:36.266243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29436954-602d-4554-9d4e-0ad064866792 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:36.274103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd5fd3-ee58-4f50-9686-ddfa04656c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:36.275011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd5fd3-ee58-4f50-9686-ddfa04656c75 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:36.275477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd5fd3-ee58-4f50-9686-ddfa04656c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:36.290515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd5fd3-ee58-4f50-9686-ddfa04656c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:36.290515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd5fd3-ee58-4f50-9686-ddfa04656c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:36.291955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd5fd3-ee58-4f50-9686-ddfa04656c75 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:36.316843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1038/2069] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:36.321003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6740d7-c4ca-4db0-8831-fd92090d9b18 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:36.338244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6740d7-c4ca-4db0-8831-fd92090d9b18 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:36.350775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0921dd3-2115-40d4-adb6-e2e224a19793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:36.351789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921dd3-2115-40d4-adb6-e2e224a19793 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:36.352128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921dd3-2115-40d4-adb6-e2e224a19793 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:36.352965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0921dd3-2115-40d4-adb6-e2e224a19793 admin admin] PATH_INFO: `/v3/projects/27c7fe7f6f1a4318a8f8fd57ba026aef/users/dda0a0f01ac544f58d50d8a9c5f2f38a/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:36.371701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0921dd3-2115-40d4-adb6-e2e224a19793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c7fe7f6f1a4318a8f8fd57ba026aef, user_id=dda0a0f01ac544f58d50d8a9c5f2f38a, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:36.374741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0921dd3-2115-40d4-adb6-e2e224a19793 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:36.531252 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1032/2070] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:36 2022] PUT /identity/v3/projects/27c7fe7f6f1a4318a8f8fd57ba026aef/users/dda0a0f01ac544f58d50d8a9c5f2f38a/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:36.543471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55c8e-7328-44a6-9138-e3aedd28be17 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:36.544028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55c8e-7328-44a6-9138-e3aedd28be17 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:36.544106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55c8e-7328-44a6-9138-e3aedd28be17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:36.579355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-12a55c8e-7328-44a6-9138-e3aedd28be17 None None] MFA Rules not processed for user `dda0a0f01ac544f58d50d8a9c5f2f38a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:36.580682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-12a55c8e-7328-44a6-9138-e3aedd28be17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:36.702712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1039/2071] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:36.706895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0921dd3-2115-40d4-adb6-e2e224a19793 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:36.848500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0921dd3-2115-40d4-adb6-e2e224a19793 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:36.883182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dcd7-3751-4fe0-bd39-61624fa53db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:36.884642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dcd7-3751-4fe0-bd39-61624fa53db6 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:36.886522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dcd7-3751-4fe0-bd39-61624fa53db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:36.895881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dcd7-3751-4fe0-bd39-61624fa53db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:36.897056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304dcd7-3751-4fe0-bd39-61624fa53db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:36.901465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304dcd7-3751-4fe0-bd39-61624fa53db6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:37.043057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1033/2072] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:36 2022] POST /identity/v3/projects => generated 369 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:37.054635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a55c8e-7328-44a6-9138-e3aedd28be17 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:37.073506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a55c8e-7328-44a6-9138-e3aedd28be17 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:37.181972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcb76e-b6fc-413a-9ea7-b0ed0b2f441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:37.182954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb76e-b6fc-413a-9ea7-b0ed0b2f441d admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:37.184102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb76e-b6fc-413a-9ea7-b0ed0b2f441d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:37.185280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb76e-b6fc-413a-9ea7-b0ed0b2f441d admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:37.186211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcb76e-b6fc-413a-9ea7-b0ed0b2f441d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:37.189018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcb76e-b6fc-413a-9ea7-b0ed0b2f441d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:37.385415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1040/2073] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:37 2022] POST /identity/v3/users => generated 291 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:37.391185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dcd7-3751-4fe0-bd39-61624fa53db6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:37.421740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dcd7-3751-4fe0-bd39-61624fa53db6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:37.440215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae52064-ac1a-4440-b719-5af0cd31608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:37.440215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae52064-ac1a-4440-b719-5af0cd31608c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:37.440612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae52064-ac1a-4440-b719-5af0cd31608c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:37.440956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae52064-ac1a-4440-b719-5af0cd31608c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:37.441767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae52064-ac1a-4440-b719-5af0cd31608c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:37.443512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae52064-ac1a-4440-b719-5af0cd31608c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:37.464842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcb76e-b6fc-413a-9ea7-b0ed0b2f441d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:37.471364 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1034/2074] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:37.477560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcb76e-b6fc-413a-9ea7-b0ed0b2f441d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:37.488290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:37.490426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:37.491013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:37.491534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] PATH_INFO: `/v3/projects/53ff4a86113049c99b7f70e5aa9f09c4/users/9915d6f3e4e345b18e60eb00d3f7f928/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:37.518076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ff4a86113049c99b7f70e5aa9f09c4, user_id=9915d6f3e4e345b18e60eb00d3f7f928, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:37.522179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:37.607173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:37.654626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1041/2075] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:37 2022] PUT /identity/v3/projects/53ff4a86113049c99b7f70e5aa9f09c4/users/9915d6f3e4e345b18e60eb00d3f7f928/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:37.661774 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae52064-ac1a-4440-b719-5af0cd31608c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:37.682024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae52064-ac1a-4440-b719-5af0cd31608c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:37.788267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:37.819433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b00ce5-1d64-4cbe-92c3-fb219fd1034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:37.820549 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00ce5-1d64-4cbe-92c3-fb219fd1034b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:37.821428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00ce5-1d64-4cbe-92c3-fb219fd1034b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:37.821715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b00ce5-1d64-4cbe-92c3-fb219fd1034b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:37.822454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b00ce5-1d64-4cbe-92c3-fb219fd1034b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:37.824747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b00ce5-1d64-4cbe-92c3-fb219fd1034b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:37.846558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:37.847522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1035/2076] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:37.848589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b00ce5-1d64-4cbe-92c3-fb219fd1034b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:37.848707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:37.865885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b00ce5-1d64-4cbe-92c3-fb219fd1034b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:37.873993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d17a2b-c9b8-4fa2-b474-02f46cfd9782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:37.875764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17a2b-c9b8-4fa2-b474-02f46cfd9782 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:37.876649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17a2b-c9b8-4fa2-b474-02f46cfd9782 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:37.877106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d17a2b-c9b8-4fa2-b474-02f46cfd9782 admin admin] PATH_INFO: `/v3/projects/53ff4a86113049c99b7f70e5aa9f09c4/users/9915d6f3e4e345b18e60eb00d3f7f928/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:37.883787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d17a2b-c9b8-4fa2-b474-02f46cfd9782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ff4a86113049c99b7f70e5aa9f09c4, user_id=9915d6f3e4e345b18e60eb00d3f7f928, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:37.886660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d17a2b-c9b8-4fa2-b474-02f46cfd9782 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:37.907711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1036/2077] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:37 2022] PUT /identity/v3/projects/53ff4a86113049c99b7f70e5aa9f09c4/users/9915d6f3e4e345b18e60eb00d3f7f928/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:37.914999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55608de7-a1e0-42ce-ab2e-aea6d4386813 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:37.915341 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55608de7-a1e0-42ce-ab2e-aea6d4386813 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:37.915603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55608de7-a1e0-42ce-ab2e-aea6d4386813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:37.958333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-55608de7-a1e0-42ce-ab2e-aea6d4386813 None None] MFA Rules not processed for user `9915d6f3e4e345b18e60eb00d3f7f928`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:37.959631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-55608de7-a1e0-42ce-ab2e-aea6d4386813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:37.978337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b732ef-8834-448d-9f4e-b47d327ff7bb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:38.091203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2adb7-08ad-44d5-bac9-56819e13eb58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:38.091901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2adb7-08ad-44d5-bac9-56819e13eb58 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:38.092192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2adb7-08ad-44d5-bac9-56819e13eb58 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:38.092474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2adb7-08ad-44d5-bac9-56819e13eb58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:38.100098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ecc2adb7-08ad-44d5-bac9-56819e13eb58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:38.112441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1037/2078] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:37 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:38.120519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55608de7-a1e0-42ce-ab2e-aea6d4386813 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:38.123901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-55608de7-a1e0-42ce-ab2e-aea6d4386813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:38.141983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55608de7-a1e0-42ce-ab2e-aea6d4386813 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:38.148181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ef067-4d14-42a0-a089-bda463a9d432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:38.148942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ef067-4d14-42a0-a089-bda463a9d432 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:38.149331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ef067-4d14-42a0-a089-bda463a9d432 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:38.149620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ef067-4d14-42a0-a089-bda463a9d432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:38.161974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ef067-4d14-42a0-a089-bda463a9d432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:38.164216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ef067-4d14-42a0-a089-bda463a9d432 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:38.202010 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1038/2079] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:38 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:38.252913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2adb7-08ad-44d5-bac9-56819e13eb58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:38.254556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2adb7-08ad-44d5-bac9-56819e13eb58 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:38.271755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1042/2080] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:38.326108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ef067-4d14-42a0-a089-bda463a9d432 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:38.347236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ef067-4d14-42a0-a089-bda463a9d432 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:38.355066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:38.356305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:38.356908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:38.358267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:38.456608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:38.458757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:38.491371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1039/2081] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:41.481175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ff961-cfed-449c-a89d-69f22a6df4c5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:41.482202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ff961-cfed-449c-a89d-69f22a6df4c5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:41.482560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ff961-cfed-449c-a89d-69f22a6df4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:41.522266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:41.540886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1e8ff961-cfed-449c-a89d-69f22a6df4c5 None None] MFA Rules not processed for user `bff41a6af0ea495a856fd5603e810d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:41.543186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1e8ff961-cfed-449c-a89d-69f22a6df4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:41.688865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1043/2082] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:41.691932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:41.693176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:41.767732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:41.810498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864ba4d-1938-4016-be65-2a325159f5bb service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:41.829539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec935df-5ad2-4702-9d69-3162576a88f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:41.830555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec935df-5ad2-4702-9d69-3162576a88f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:41.831131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec935df-5ad2-4702-9d69-3162576a88f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:41.831630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec935df-5ad2-4702-9d69-3162576a88f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:41.833239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec935df-5ad2-4702-9d69-3162576a88f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:41.835421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec935df-5ad2-4702-9d69-3162576a88f4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:42.025745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1040/2083] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:41 2022] POST /identity/v3/projects => generated 385 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:42.029516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ff961-cfed-449c-a89d-69f22a6df4c5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:42.050611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ff961-cfed-449c-a89d-69f22a6df4c5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:42.178967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5029b-8c4b-4e99-abf6-87505e23da16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:42.179704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5029b-8c4b-4e99-abf6-87505e23da16 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:42.180014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5029b-8c4b-4e99-abf6-87505e23da16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:42.180376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5029b-8c4b-4e99-abf6-87505e23da16 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:42.181087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5029b-8c4b-4e99-abf6-87505e23da16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:42.183082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5029b-8c4b-4e99-abf6-87505e23da16 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:42.284376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1044/2084] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:42 2022] POST /identity/v3/users => generated 299 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:42.288336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec935df-5ad2-4702-9d69-3162576a88f4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:42.299738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec935df-5ad2-4702-9d69-3162576a88f4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:42.304688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a72024-e96f-4d30-b97d-87e20e30a75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:42.305369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72024-e96f-4d30-b97d-87e20e30a75d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:42.305657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72024-e96f-4d30-b97d-87e20e30a75d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:42.305966 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a72024-e96f-4d30-b97d-87e20e30a75d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:42.306729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a72024-e96f-4d30-b97d-87e20e30a75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:42.309744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a72024-e96f-4d30-b97d-87e20e30a75d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:42.321203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1041/2085] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:42.326283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5029b-8c4b-4e99-abf6-87505e23da16 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:42.336798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5029b-8c4b-4e99-abf6-87505e23da16 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:42.342451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b17ab4-84f5-499c-9e6e-268a747859be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:42.343594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b17ab4-84f5-499c-9e6e-268a747859be admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:42.343892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b17ab4-84f5-499c-9e6e-268a747859be admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:42.344177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b17ab4-84f5-499c-9e6e-268a747859be admin admin] PATH_INFO: `/v3/projects/d768772b39f840328a0edad241b87b95/users/386071149f824b5d94dfa0e27249f3a3/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:42.362984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b17ab4-84f5-499c-9e6e-268a747859be admin admin] RBAC: Authorizing `identity:create_grant(project_id=d768772b39f840328a0edad241b87b95, user_id=386071149f824b5d94dfa0e27249f3a3, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:42.365712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b17ab4-84f5-499c-9e6e-268a747859be admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:42.503441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1045/2086] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:42 2022] PUT /identity/v3/projects/d768772b39f840328a0edad241b87b95/users/386071149f824b5d94dfa0e27249f3a3/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:42.507330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a72024-e96f-4d30-b97d-87e20e30a75d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:42.517287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a72024-e96f-4d30-b97d-87e20e30a75d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:42.588895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ed34f-2cfb-4a3d-b17f-ecd7fa954217 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:42.589183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ed34f-2cfb-4a3d-b17f-ecd7fa954217 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:42.589691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ed34f-2cfb-4a3d-b17f-ecd7fa954217 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:42.591099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1046/2087] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:55:42.592006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7cd71-9372-480a-b64d-b1eadc646d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:42.592658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7cd71-9372-480a-b64d-b1eadc646d65 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:42.592948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7cd71-9372-480a-b64d-b1eadc646d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:42.598547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7cd71-9372-480a-b64d-b1eadc646d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:42.598547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7cd71-9372-480a-b64d-b1eadc646d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:42.598547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7cd71-9372-480a-b64d-b1eadc646d65 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:42.600982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842db86-0931-4891-81e5-1fdb12cc256d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:42.603929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842db86-0931-4891-81e5-1fdb12cc256d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:42.604641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842db86-0931-4891-81e5-1fdb12cc256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:42.611641 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1042/2088] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:42.612836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7cd71-9372-480a-b64d-b1eadc646d65 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:42.628506 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7cd71-9372-480a-b64d-b1eadc646d65 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:42.639797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:42.640724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1842db86-0931-4891-81e5-1fdb12cc256d None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:42.640912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:42.641211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:42.641489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] PATH_INFO: `/v3/projects/d768772b39f840328a0edad241b87b95/users/386071149f824b5d94dfa0e27249f3a3/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:42.645993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1842db86-0931-4891-81e5-1fdb12cc256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:42.646128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d768772b39f840328a0edad241b87b95, user_id=386071149f824b5d94dfa0e27249f3a3, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:42.646919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:42.674076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1043/2089] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:42 2022] PUT /identity/v3/projects/d768772b39f840328a0edad241b87b95/users/386071149f824b5d94dfa0e27249f3a3/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:42.743955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1047/2090] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:55:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:42.759331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:42.761142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:42.850202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17cedf-3da0-4da3-b642-1de29c12c9ae admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:42.862905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d89b8-9364-4ad6-b9f2-7574615109ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:42.864886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d89b8-9364-4ad6-b9f2-7574615109ad service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:42.865315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d89b8-9364-4ad6-b9f2-7574615109ad service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:42.865808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d89b8-9364-4ad6-b9f2-7574615109ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:42.867739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842db86-0931-4891-81e5-1fdb12cc256d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:42.870150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1842db86-0931-4891-81e5-1fdb12cc256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:42.882114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d89b8-9364-4ad6-b9f2-7574615109ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:42.884918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d89b8-9364-4ad6-b9f2-7574615109ad service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:42.892920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842db86-0931-4891-81e5-1fdb12cc256d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:42.926440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1044/2091] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:43.018223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:43.020621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:43.021174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:43.021615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] PATH_INFO: `/v3/users/8e90cd544b0745809f118964cbdc66bb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:43.024002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e90cd544b0745809f118964cbdc66bb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:43.026867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:43.032989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:43.366726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e90cd544b0745809f118964cbdc66bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:43.397262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e90cd544b0745809f118964cbdc66bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:43.398030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e90cd544b0745809f118964cbdc66bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:43.398731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e90cd544b0745809f118964cbdc66bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:43.417681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c1060b0b-36f7-4bef-9172-9d08b7af04db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e90cd544b0745809f118964cbdc66bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:43.547145 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1048/2092] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:42 2022] DELETE /identity/v3/users/8e90cd544b0745809f118964cbdc66bb => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:44.134715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d89b8-9364-4ad6-b9f2-7574615109ad service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:44.148232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d89b8-9364-4ad6-b9f2-7574615109ad service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:44.173976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:44.236865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:44.237809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:44.238101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:44.238599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] PATH_INFO: `/v3/projects/a5644a8a1ae540e2b260c3d1bb2bea0f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:44.240298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5644a8a1ae540e2b260c3d1bb2bea0f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:44.242939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:44.425520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5644a8a1ae540e2b260c3d1bb2bea0f', 'request_id': 'req-1ddf3292-1ff4-41e1-964d-dae3192aea54'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:44.431709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5644a8a1ae540e2b260c3d1bb2bea0f', 'request_id': 'req-1ddf3292-1ff4-41e1-964d-dae3192aea54'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:44.437161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5644a8a1ae540e2b260c3d1bb2bea0f', 'request_id': 'req-1ddf3292-1ff4-41e1-964d-dae3192aea54'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:44.599914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] The token cache is being invalidate because project a5644a8a1ae540e2b260c3d1bb2bea0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:44.600203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:44.601288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:44.603066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1045/2093] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:44 2022] DELETE /identity/v3/projects/a5644a8a1ae540e2b260c3d1bb2bea0f => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:44.608321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920df4e-7b10-40d6-babc-89826bcc5947 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:44.608824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920df4e-7b10-40d6-babc-89826bcc5947 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:44.608906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920df4e-7b10-40d6-babc-89826bcc5947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:44.640821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9920df4e-7b10-40d6-babc-89826bcc5947 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:44.642286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9920df4e-7b10-40d6-babc-89826bcc5947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:44.785121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:44.787504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:44.794712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1049/2094] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:44.803353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fe33c-a79c-46d0-be82-607bc0123e2a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:44.806205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fe33c-a79c-46d0-be82-607bc0123e2a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:44.806598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fe33c-a79c-46d0-be82-607bc0123e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:44.811922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf3292-1ff4-41e1-964d-dae3192aea54 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:44.823897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:44.825374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:44.825845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:44.826279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] PATH_INFO: `/v3/users/ba6099c1f81a42c89ccbc33e07f9e26a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:44.827753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6099c1f81a42c89ccbc33e07f9e26a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:44.829899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:44.837173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d37fe33c-a79c-46d0-be82-607bc0123e2a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:44.840364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d37fe33c-a79c-46d0-be82-607bc0123e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:44.868932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1050/2095] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:44.870292 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fe33c-a79c-46d0-be82-607bc0123e2a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:44.888733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fe33c-a79c-46d0-be82-607bc0123e2a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:44.904625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:44.905480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:44.905773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:44.906149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:44.908139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:44.908466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:44.914636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:44.937405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:44.941252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1051/2096] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:55:44.946027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:44.958964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912aab-b12f-4f3d-b1d6-9f0d517ad4fc admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:44.964773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103db05e-685c-4522-965a-c56831579ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:44.971163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103db05e-685c-4522-965a-c56831579ccd admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:44.971163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103db05e-685c-4522-965a-c56831579ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:44.971163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103db05e-685c-4522-965a-c56831579ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:44.971163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103db05e-685c-4522-965a-c56831579ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:44.971163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103db05e-685c-4522-965a-c56831579ccd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:45.092729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6099c1f81a42c89ccbc33e07f9e26a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:45.115387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1052/2097] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:44 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:45.118413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103db05e-685c-4522-965a-c56831579ccd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:45.134632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6099c1f81a42c89ccbc33e07f9e26a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:45.135430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6099c1f81a42c89ccbc33e07f9e26a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:45.136113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6099c1f81a42c89ccbc33e07f9e26a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:45.151497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6099c1f81a42c89ccbc33e07f9e26a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:45.208858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103db05e-685c-4522-965a-c56831579ccd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:45.215406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f5329-1bae-4fb4-9a54-fa324c062e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:45.216491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5329-1bae-4fb4-9a54-fa324c062e2b admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:45.216794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5329-1bae-4fb4-9a54-fa324c062e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:45.217147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f5329-1bae-4fb4-9a54-fa324c062e2b admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:45.217865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5329-1bae-4fb4-9a54-fa324c062e2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:45.219943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f5329-1bae-4fb4-9a54-fa324c062e2b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:45.326395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1046/2098] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:44 2022] DELETE /identity/v3/users/ba6099c1f81a42c89ccbc33e07f9e26a => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:45.327281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:45.344138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:45.398555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1053/2099] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:45 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:45.400055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b7de7-fa11-45ea-a834-1320f90bbde3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:45.400362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b7de7-fa11-45ea-a834-1320f90bbde3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:45.400648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b7de7-fa11-45ea-a834-1320f90bbde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:45.511566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-be4b7de7-fa11-45ea-a834-1320f90bbde3 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:45.516263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86969b4-08b2-46e9-b30c-278fa16d6208 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:45.524734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-be4b7de7-fa11-45ea-a834-1320f90bbde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:45.536468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:45.542788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:45.544455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:45.547656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] PATH_INFO: `/v3/users/dda0a0f01ac544f58d50d8a9c5f2f38a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:45.552468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dda0a0f01ac544f58d50d8a9c5f2f38a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:45.574536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1054/2100] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:45.574536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b7de7-fa11-45ea-a834-1320f90bbde3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:45.575560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-be4b7de7-fa11-45ea-a834-1320f90bbde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:45.581435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:45.607426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b7de7-fa11-45ea-a834-1320f90bbde3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:45.626608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:45.627759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:45.628253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:45.628750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:45.630140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:45.658503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:45.682888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1055/2101] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:45.685562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:45.688510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:45.890450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbaa29-afdc-4f49-9b9d-76f9a3a97885 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:45.899724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273209-f842-4875-b38c-3c9d3ba4b284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:45.900905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273209-f842-4875-b38c-3c9d3ba4b284 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:45.903141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273209-f842-4875-b38c-3c9d3ba4b284 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:45.903638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273209-f842-4875-b38c-3c9d3ba4b284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:45.937239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0273209-f842-4875-b38c-3c9d3ba4b284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:45.938739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0273209-f842-4875-b38c-3c9d3ba4b284 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:45.951246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1056/2102] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:45.952602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273209-f842-4875-b38c-3c9d3ba4b284 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:45.975349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273209-f842-4875-b38c-3c9d3ba4b284 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:45.981643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2c8ce-61b0-4189-a6ed-5a6c91769e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:45.982449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c8ce-61b0-4189-a6ed-5a6c91769e56 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:45.982855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c8ce-61b0-4189-a6ed-5a6c91769e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:45.983238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2c8ce-61b0-4189-a6ed-5a6c91769e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:45.984109 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2c8ce-61b0-4189-a6ed-5a6c91769e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:45.986711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2c8ce-61b0-4189-a6ed-5a6c91769e56 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:46.004942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1057/2103] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:46.006332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2c8ce-61b0-4189-a6ed-5a6c91769e56 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:46.030338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2c8ce-61b0-4189-a6ed-5a6c91769e56 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:46.030338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5e26c-4896-46b6-86af-c65406f8d6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:46.030338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5e26c-4896-46b6-86af-c65406f8d6b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:46.030338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5e26c-4896-46b6-86af-c65406f8d6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:46.030759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5e26c-4896-46b6-86af-c65406f8d6b3 admin admin] PATH_INFO: `/v3/projects/c35d3e93d36040bba1bbb873e61d239a/users/1157b91a306540dd98a072b1ea4941b2/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:46.046255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dda0a0f01ac544f58d50d8a9c5f2f38a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:46.052716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5e26c-4896-46b6-86af-c65406f8d6b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35d3e93d36040bba1bbb873e61d239a, user_id=1157b91a306540dd98a072b1ea4941b2, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:46.060905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5e26c-4896-46b6-86af-c65406f8d6b3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:46.068864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dda0a0f01ac544f58d50d8a9c5f2f38a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:46.069667 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dda0a0f01ac544f58d50d8a9c5f2f38a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:46.070570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dda0a0f01ac544f58d50d8a9c5f2f38a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:46.082215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5549f42b-c38f-431f-8b95-65e356b52c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dda0a0f01ac544f58d50d8a9c5f2f38a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:46.179816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1058/2104] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:46 2022] PUT /identity/v3/projects/c35d3e93d36040bba1bbb873e61d239a/users/1157b91a306540dd98a072b1ea4941b2/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:46.185525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b818-f167-45fa-b017-bb7af9f500c3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:46.186054 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b818-f167-45fa-b017-bb7af9f500c3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:46.186516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b818-f167-45fa-b017-bb7af9f500c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:46.207470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5934b818-f167-45fa-b017-bb7af9f500c3 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:46.212492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5934b818-f167-45fa-b017-bb7af9f500c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:46.221519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1047/2105] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:45 2022] DELETE /identity/v3/users/dda0a0f01ac544f58d50d8a9c5f2f38a => generated 0 bytes in 894 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:46.225866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca93ab-8cc8-4fed-9727-79925c6c2890 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:46.226170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca93ab-8cc8-4fed-9727-79925c6c2890 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:46.226454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca93ab-8cc8-4fed-9727-79925c6c2890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:46.255946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-24ca93ab-8cc8-4fed-9727-79925c6c2890 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:46.257653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-24ca93ab-8cc8-4fed-9727-79925c6c2890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:46.308637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1059/2106] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:46.319193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b818-f167-45fa-b017-bb7af9f500c3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:46.322475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5934b818-f167-45fa-b017-bb7af9f500c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:46.366699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1048/2107] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:46.375419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca93ab-8cc8-4fed-9727-79925c6c2890 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:46.383260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-24ca93ab-8cc8-4fed-9727-79925c6c2890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:46.450272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b818-f167-45fa-b017-bb7af9f500c3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:46.482818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca93ab-8cc8-4fed-9727-79925c6c2890 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:46.486843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:46.487693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:46.487987 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:46.488265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:46.501942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:46.503536 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:46.523666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1049/2108] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:46.528130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:46.528785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:46.529175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:46.529573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:46.567617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:46.572328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:46.644319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1060/2109] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:47.063073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:47.064807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:47.079465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926767ba-aaf8-4f1e-a09a-d65e5edd69c3 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:47.085802 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:47.093100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:47.093395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:47.093690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] PATH_INFO: `/v3/projects/5031fbad9e18426484f5fffcd9c01bc4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:47.096943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5031fbad9e18426484f5fffcd9c01bc4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:47.099660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:47.268479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:47.290561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5031fbad9e18426484f5fffcd9c01bc4', 'request_id': 'req-227f395d-c7e3-453b-a619-e510eb4c56b6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.297566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5031fbad9e18426484f5fffcd9c01bc4', 'request_id': 'req-227f395d-c7e3-453b-a619-e510eb4c56b6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.303456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5031fbad9e18426484f5fffcd9c01bc4', 'request_id': 'req-227f395d-c7e3-453b-a619-e510eb4c56b6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.353856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:47.356249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:47.374393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac2729-3a52-47fd-9811-643f53ac58ad service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:47.462874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] The token cache is being invalidate because project 5031fbad9e18426484f5fffcd9c01bc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:47.463080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.464915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.467736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1050/2110] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:47 2022] DELETE /identity/v3/projects/5031fbad9e18426484f5fffcd9c01bc4 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:47.477586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:47.480790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:47.556441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:47.556441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:47.556441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:47.556441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] PATH_INFO: `/v3/projects/27c7fe7f6f1a4318a8f8fd57ba026aef` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:47.556819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c7fe7f6f1a4318a8f8fd57ba026aef)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:47.556819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:47.645287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f395d-c7e3-453b-a619-e510eb4c56b6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:47.655158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:47.655158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:47.655158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:47.655158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] PATH_INFO: `/v3/users/e238bfa601eb4470b99399288984b629` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:47.656588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e238bfa601eb4470b99399288984b629)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:47.659671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:47.743180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c7fe7f6f1a4318a8f8fd57ba026aef', 'request_id': 'req-6c5da5bc-94f9-4612-9685-df4781984599'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.770146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c7fe7f6f1a4318a8f8fd57ba026aef', 'request_id': 'req-6c5da5bc-94f9-4612-9685-df4781984599'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.889880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c7fe7f6f1a4318a8f8fd57ba026aef', 'request_id': 'req-6c5da5bc-94f9-4612-9685-df4781984599'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.909857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] The token cache is being invalidate because project 27c7fe7f6f1a4318a8f8fd57ba026aef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:47.910518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.911262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.913019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1061/2111] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:47 2022] DELETE /identity/v3/projects/27c7fe7f6f1a4318a8f8fd57ba026aef => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:47.916982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:47.919043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:47.949457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e238bfa601eb4470b99399288984b629'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.966233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e238bfa601eb4470b99399288984b629'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.967173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e238bfa601eb4470b99399288984b629'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.968049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e238bfa601eb4470b99399288984b629'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:47.978909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e238bfa601eb4470b99399288984b629'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:48.023205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da5bc-94f9-4612-9685-df4781984599 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:48.030029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:48.030925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:48.031217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:48.031534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] PATH_INFO: `/v3/users/9915d6f3e4e345b18e60eb00d3f7f928` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:48.032775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9915d6f3e4e345b18e60eb00d3f7f928)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:48.034727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:48.216347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1051/2112] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:47 2022] DELETE /identity/v3/users/e238bfa601eb4470b99399288984b629 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:48.611970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9915d6f3e4e345b18e60eb00d3f7f928'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:48.663201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9915d6f3e4e345b18e60eb00d3f7f928'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:48.666767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9915d6f3e4e345b18e60eb00d3f7f928'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:48.668738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9915d6f3e4e345b18e60eb00d3f7f928'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:48.698627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9915d6f3e4e345b18e60eb00d3f7f928'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:48.826713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1062/2113] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:47 2022] DELETE /identity/v3/users/9915d6f3e4e345b18e60eb00d3f7f928 => generated 0 bytes in 906 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:49.247639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:49.249328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:49.361676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678452-347c-493a-92b2-502ef2dc73cd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:49.440132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:49.443651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:49.443945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:49.444234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] PATH_INFO: `/v3/projects/8a1fe94ee8744e6da47af5c719a6b413` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:49.445849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1fe94ee8744e6da47af5c719a6b413)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:49.447831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:49.695367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:49.697046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:49.719914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1fe94ee8744e6da47af5c719a6b413', 'request_id': 'req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:49.721702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8084f8-3f5d-40ba-babb-2ea1ed8b74aa admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:49.740638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1fe94ee8744e6da47af5c719a6b413', 'request_id': 'req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:49.753688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1fe94ee8744e6da47af5c719a6b413', 'request_id': 'req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:49.845942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:49.846998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:49.847307 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:49.847600 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] PATH_INFO: `/v3/projects/53ff4a86113049c99b7f70e5aa9f09c4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:49.848819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ff4a86113049c99b7f70e5aa9f09c4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:49.850830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:49.905149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:49.914749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] The token cache is being invalidate because project 8a1fe94ee8744e6da47af5c719a6b413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:49.915179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:49.916275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:49.921242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1052/2114] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:49 2022] DELETE /identity/v3/projects/8a1fe94ee8744e6da47af5c719a6b413 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:49.922365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:49.923907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:49.941780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b20b-f3ae-4c50-9e05-b3538aaf02a5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:49.947938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:49.949182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:49.949479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:49.949777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] PATH_INFO: `/v3/users/0d71281db6de4d82ad0b75f51042dca9` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:49.951101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d71281db6de4d82ad0b75f51042dca9)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:49.953635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:50.033944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ff4a86113049c99b7f70e5aa9f09c4', 'request_id': 'req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.042976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ff4a86113049c99b7f70e5aa9f09c4', 'request_id': 'req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.210736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ff4a86113049c99b7f70e5aa9f09c4', 'request_id': 'req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.230269 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] The token cache is being invalidate because project 53ff4a86113049c99b7f70e5aa9f09c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:50.230565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.231260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-aa9db7ec-2074-47b7-9511-5e976ac9a5b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.234055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d71281db6de4d82ad0b75f51042dca9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.234845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1063/2115] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:49 2022] DELETE /identity/v3/projects/53ff4a86113049c99b7f70e5aa9f09c4 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:50.239904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf88e1-874c-4ad8-bbeb-8d3a834403d6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:50.240203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf88e1-874c-4ad8-bbeb-8d3a834403d6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:50.240523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf88e1-874c-4ad8-bbeb-8d3a834403d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:50.267130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d71281db6de4d82ad0b75f51042dca9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.267130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d71281db6de4d82ad0b75f51042dca9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.267130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d71281db6de4d82ad0b75f51042dca9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.271779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d71281db6de4d82ad0b75f51042dca9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:50.298370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9ddf88e1-874c-4ad8-bbeb-8d3a834403d6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:50.299713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9ddf88e1-874c-4ad8-bbeb-8d3a834403d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:50.388655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1064/2116] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:50.395994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeca15d-96be-4c28-897b-f7f131c64230 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:50.396288 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeca15d-96be-4c28-897b-f7f131c64230 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:50.396574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeca15d-96be-4c28-897b-f7f131c64230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:50.412473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1053/2117] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:49 2022] DELETE /identity/v3/users/0d71281db6de4d82ad0b75f51042dca9 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:50.431196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0eeca15d-96be-4c28-897b-f7f131c64230 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:50.433579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0eeca15d-96be-4c28-897b-f7f131c64230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:50.515488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1065/2118] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:50.517972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:50.529298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7b32-e66c-4e66-b455-a35820d9a535 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:50.534965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:50.535912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:50.536405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:50.536822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:50.537723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:50.538400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:50.540247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:50.556478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1054/2119] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:55:50.561767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeca15d-96be-4c28-897b-f7f131c64230 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:50.572649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeca15d-96be-4c28-897b-f7f131c64230 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:50.578708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:50.579365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:50.580560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:50.580560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:50.582135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:50.584637 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:50.716156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1066/2120] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:50 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:50.720115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:50.734548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65214dd1-bc06-41da-87f8-eedbb29b008d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:50.816349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8de470-ea7f-42a2-a7b2-46126cbe6552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:50.817897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8de470-ea7f-42a2-a7b2-46126cbe6552 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:50.818253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8de470-ea7f-42a2-a7b2-46126cbe6552 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:50.818652 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8de470-ea7f-42a2-a7b2-46126cbe6552 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:50.820150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8de470-ea7f-42a2-a7b2-46126cbe6552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:50.822393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8de470-ea7f-42a2-a7b2-46126cbe6552 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:50.923968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:50.925986 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:50.954655 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33497e1e-a8df-42e8-acad-6ac519c52ce3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:50.961144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:50.962104 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:50.962396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:50.962675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] PATH_INFO: `/v3/projects/ff3ff4087b5e4a1eb2036e0b40da3141` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:50.965063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff3ff4087b5e4a1eb2036e0b40da3141)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:50.967653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:50.998696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1055/2121] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:50 2022] POST /identity/v3/users => generated 294 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:51.002701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8de470-ea7f-42a2-a7b2-46126cbe6552 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:51.015701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8de470-ea7f-42a2-a7b2-46126cbe6552 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:51.028104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813267d-8a07-49de-84c2-9c8645aebe2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:51.028104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813267d-8a07-49de-84c2-9c8645aebe2b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.028104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813267d-8a07-49de-84c2-9c8645aebe2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.028104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813267d-8a07-49de-84c2-9c8645aebe2b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.029027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813267d-8a07-49de-84c2-9c8645aebe2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:51.029027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813267d-8a07-49de-84c2-9c8645aebe2b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:51.044350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1056/2122] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:51.045904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813267d-8a07-49de-84c2-9c8645aebe2b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:51.116978 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813267d-8a07-49de-84c2-9c8645aebe2b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:51.121133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:51.154042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff3ff4087b5e4a1eb2036e0b40da3141', 'request_id': 'req-25c61090-b192-432a-89b1-d32c58a11bc4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:51.162408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff3ff4087b5e4a1eb2036e0b40da3141', 'request_id': 'req-25c61090-b192-432a-89b1-d32c58a11bc4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:51.229603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14019b34-3edb-4897-b137-0dcabcf85cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:51.230469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019b34-3edb-4897-b137-0dcabcf85cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.232220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019b34-3edb-4897-b137-0dcabcf85cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.232529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019b34-3edb-4897-b137-0dcabcf85cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.234035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14019b34-3edb-4897-b137-0dcabcf85cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:51.236424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14019b34-3edb-4897-b137-0dcabcf85cb3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:51.254909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1057/2123] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:51.255911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14019b34-3edb-4897-b137-0dcabcf85cb3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:51.274579 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14019b34-3edb-4897-b137-0dcabcf85cb3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:51.276719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c7f31-975e-4a25-9e5c-ef045cd51259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:51.277701 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c7f31-975e-4a25-9e5c-ef045cd51259 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.278061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c7f31-975e-4a25-9e5c-ef045cd51259 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.278432 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c7f31-975e-4a25-9e5c-ef045cd51259 admin admin] PATH_INFO: `/v3/projects/d0ba8f444fc247f98273872bc7dab144/users/bdda1d2d0cbb46dfb50c220ffcaecdaa/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.300022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff3ff4087b5e4a1eb2036e0b40da3141', 'request_id': 'req-25c61090-b192-432a-89b1-d32c58a11bc4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:51.308975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c7f31-975e-4a25-9e5c-ef045cd51259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ba8f444fc247f98273872bc7dab144, user_id=bdda1d2d0cbb46dfb50c220ffcaecdaa, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:51.312831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c7f31-975e-4a25-9e5c-ef045cd51259 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:51.320527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] The token cache is being invalidate because project ff3ff4087b5e4a1eb2036e0b40da3141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:51.320527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:51.323882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-25c61090-b192-432a-89b1-d32c58a11bc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:51.331351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1067/2124] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:50 2022] DELETE /identity/v3/projects/ff3ff4087b5e4a1eb2036e0b40da3141 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:51.345447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59caea4a-d9ab-4e96-8f49-255dbb1e10d3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.350632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59caea4a-d9ab-4e96-8f49-255dbb1e10d3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.350632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59caea4a-d9ab-4e96-8f49-255dbb1e10d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.383096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-59caea4a-d9ab-4e96-8f49-255dbb1e10d3 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:51.384759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-59caea4a-d9ab-4e96-8f49-255dbb1e10d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:51.395736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1068/2125] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:51.411088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ce42a-b86e-4e15-ab4e-3cf3ae62ebd6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.411088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ce42a-b86e-4e15-ab4e-3cf3ae62ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.411088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ce42a-b86e-4e15-ab4e-3cf3ae62ebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.450525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1058/2126] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:51 2022] PUT /identity/v3/projects/d0ba8f444fc247f98273872bc7dab144/users/bdda1d2d0cbb46dfb50c220ffcaecdaa/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:51.450525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bdc1a-2498-4c52-8824-de47d655175b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.450752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-636ce42a-b86e-4e15-ab4e-3cf3ae62ebd6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:51.450842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bdc1a-2498-4c52-8824-de47d655175b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.450927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-636ce42a-b86e-4e15-ab4e-3cf3ae62ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:51.451105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bdc1a-2498-4c52-8824-de47d655175b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.516415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3b6bdc1a-2498-4c52-8824-de47d655175b None None] MFA Rules not processed for user `bdda1d2d0cbb46dfb50c220ffcaecdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:51.525536 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3b6bdc1a-2498-4c52-8824-de47d655175b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:51.589686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:51.597612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1069/2127] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:51.601952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ce42a-b86e-4e15-ab4e-3cf3ae62ebd6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:51.648042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ce42a-b86e-4e15-ab4e-3cf3ae62ebd6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:51.653146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:51.653721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.654086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.654427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.655218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:55:51.655575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:51.657305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:51.675334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1070/2128] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:55:51.686884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:51.699625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1059/2129] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:51.700842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bdc1a-2498-4c52-8824-de47d655175b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:51.702424 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3b6bdc1a-2498-4c52-8824-de47d655175b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:51.702795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155495d-1c16-4a14-bfa1-cafebaff5d81 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:51.714051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5166d-284a-477f-baa4-dfb8898cbd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:51.714983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5166d-284a-477f-baa4-dfb8898cbd31 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.715472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5166d-284a-477f-baa4-dfb8898cbd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.716037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5166d-284a-477f-baa4-dfb8898cbd31 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.716918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5166d-284a-477f-baa4-dfb8898cbd31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:51.719057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5166d-284a-477f-baa4-dfb8898cbd31 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:51.722151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bdc1a-2498-4c52-8824-de47d655175b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:51.728827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4dd3-9cbf-4ab6-b05b-5e3f9d191557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:51.738649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4dd3-9cbf-4ab6-b05b-5e3f9d191557 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:51.738649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4dd3-9cbf-4ab6-b05b-5e3f9d191557 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:51.738649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb4dd3-9cbf-4ab6-b05b-5e3f9d191557 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:51.738649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4dd3-9cbf-4ab6-b05b-5e3f9d191557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:51.738649 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb4dd3-9cbf-4ab6-b05b-5e3f9d191557 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:51.842748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1071/2130] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:51 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:51.844389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5166d-284a-477f-baa4-dfb8898cbd31 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:51.881606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1060/2131] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:51 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:51.901009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4dd3-9cbf-4ab6-b05b-5e3f9d191557 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:51.935042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5166d-284a-477f-baa4-dfb8898cbd31 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:51.995230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb4dd3-9cbf-4ab6-b05b-5e3f9d191557 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:52.006866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96abc4-fe8f-4f8b-8a56-d7872719609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.007656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96abc4-fe8f-4f8b-8a56-d7872719609a admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.008019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96abc4-fe8f-4f8b-8a56-d7872719609a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.008226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96abc4-fe8f-4f8b-8a56-d7872719609a admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.009441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96abc4-fe8f-4f8b-8a56-d7872719609a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.010952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96abc4-fe8f-4f8b-8a56-d7872719609a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.023775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26beb9-430f-4ad7-9be9-baf49387c51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.026881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26beb9-430f-4ad7-9be9-baf49387c51e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.038437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26beb9-430f-4ad7-9be9-baf49387c51e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.038892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26beb9-430f-4ad7-9be9-baf49387c51e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.039828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26beb9-430f-4ad7-9be9-baf49387c51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.042552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26beb9-430f-4ad7-9be9-baf49387c51e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.331736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1072/2132] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:51 2022] POST /identity/v3/users => generated 293 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:52.334748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1061/2133] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:51 2022] POST /identity/v3/users => generated 294 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:52.334748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96abc4-fe8f-4f8b-8a56-d7872719609a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:52.336426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9124-99ad-4735-8a90-2f0b069e5b2b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.336731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9124-99ad-4735-8a90-2f0b069e5b2b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.337024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9124-99ad-4735-8a90-2f0b069e5b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.348512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96abc4-fe8f-4f8b-8a56-d7872719609a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:52.354925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ce777-30a0-41a8-bb8f-3f742f75b173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.355958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce777-30a0-41a8-bb8f-3f742f75b173 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.356508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce777-30a0-41a8-bb8f-3f742f75b173 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.356798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ce777-30a0-41a8-bb8f-3f742f75b173 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.357537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ce777-30a0-41a8-bb8f-3f742f75b173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.359514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ce777-30a0-41a8-bb8f-3f742f75b173 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.381714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:52.382564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1062/2134] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:52.383910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ce777-30a0-41a8-bb8f-3f742f75b173 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:52.402023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ce777-30a0-41a8-bb8f-3f742f75b173 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:52.402851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c3eb9124-99ad-4735-8a90-2f0b069e5b2b None None] MFA Rules not processed for user `386071149f824b5d94dfa0e27249f3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:52.404664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c3eb9124-99ad-4735-8a90-2f0b069e5b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:52.408226 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04cbce-e8fc-4f3f-bbb8-6344fefc5f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.409210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04cbce-e8fc-4f3f-bbb8-6344fefc5f04 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.410237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04cbce-e8fc-4f3f-bbb8-6344fefc5f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.410697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04cbce-e8fc-4f3f-bbb8-6344fefc5f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.411612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04cbce-e8fc-4f3f-bbb8-6344fefc5f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.413698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04cbce-e8fc-4f3f-bbb8-6344fefc5f04 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.435912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1063/2135] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:52.437131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04cbce-e8fc-4f3f-bbb8-6344fefc5f04 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:52.449189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04cbce-e8fc-4f3f-bbb8-6344fefc5f04 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:52.466538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81971ff8-02b3-4ca8-8e73-fa6be4f7667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.467884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81971ff8-02b3-4ca8-8e73-fa6be4f7667f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.468102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81971ff8-02b3-4ca8-8e73-fa6be4f7667f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.468244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81971ff8-02b3-4ca8-8e73-fa6be4f7667f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.468995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81971ff8-02b3-4ca8-8e73-fa6be4f7667f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.470643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81971ff8-02b3-4ca8-8e73-fa6be4f7667f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.513153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1064/2136] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:52.516287 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81971ff8-02b3-4ca8-8e73-fa6be4f7667f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:52.541631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81971ff8-02b3-4ca8-8e73-fa6be4f7667f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:52.553941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.554634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.556466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.556466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.558225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.563241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.588909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1073/2137] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:52.589233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1065/2138] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:52.591577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9124-99ad-4735-8a90-2f0b069e5b2b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:52.594179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:52.598086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:52.609628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9124-99ad-4735-8a90-2f0b069e5b2b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:52.616842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6984b-7470-4a70-97e8-fffd68115a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.617761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6984b-7470-4a70-97e8-fffd68115a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.619256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6984b-7470-4a70-97e8-fffd68115a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.619601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6984b-7470-4a70-97e8-fffd68115a45 admin admin] PATH_INFO: `/v3/projects/febe1f57973d4c5cbd2cb66b597e5b9f/users/c305307caa7e46fb913b9a71478bb5a3/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.643747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6984b-7470-4a70-97e8-fffd68115a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febe1f57973d4c5cbd2cb66b597e5b9f, user_id=c305307caa7e46fb913b9a71478bb5a3, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.647099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6984b-7470-4a70-97e8-fffd68115a45 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.753936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09227fcf-2bba-43a4-b8dc-73dd083875f0 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:52.757656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1074/2139] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:52 2022] PUT /identity/v3/projects/febe1f57973d4c5cbd2cb66b597e5b9f/users/c305307caa7e46fb913b9a71478bb5a3/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:52.758943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6984b-7470-4a70-97e8-fffd68115a45 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:52.828692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74f65e-2554-4830-9db8-9c33a05d01b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f18d0f5f634d84a99fc3e46963a8b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.829520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74f65e-2554-4830-9db8-9c33a05d01b3 service glance] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.829820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74f65e-2554-4830-9db8-9c33a05d01b3 service glance] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.830108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74f65e-2554-4830-9db8-9c33a05d01b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.833830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9a74f65e-2554-4830-9db8-9c33a05d01b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:52.891197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6984b-7470-4a70-97e8-fffd68115a45 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:52.900923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9f200-7653-40e8-92f6-c9dc8b405c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:52.902788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9f200-7653-40e8-92f6-c9dc8b405c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.903815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9f200-7653-40e8-92f6-c9dc8b405c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.904320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9f200-7653-40e8-92f6-c9dc8b405c57 admin admin] PATH_INFO: `/v3/projects/fbd05f8be6b0477cbb1ff0bc8b6e12c5/users/ee487075fd944918b01a158eeef811dc/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:52.933419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9f200-7653-40e8-92f6-c9dc8b405c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd05f8be6b0477cbb1ff0bc8b6e12c5, user_id=ee487075fd944918b01a158eeef811dc, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.937156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9f200-7653-40e8-92f6-c9dc8b405c57 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.966974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74f65e-2554-4830-9db8-9c33a05d01b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:52.967736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74f65e-2554-4830-9db8-9c33a05d01b3 service glance] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:52.983200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1066/2140] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:52.984932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5a81f-73d6-4d0c-a5a0-cfac471f43e5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:52.985405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5a81f-73d6-4d0c-a5a0-cfac471f43e5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:52.985839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5a81f-73d6-4d0c-a5a0-cfac471f43e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.028007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1075/2141] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:52 2022] PUT /identity/v3/projects/fbd05f8be6b0477cbb1ff0bc8b6e12c5/users/ee487075fd944918b01a158eeef811dc/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:53.044766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321696d5-c86d-4b40-a615-d230ef3a0101 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:53.047547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321696d5-c86d-4b40-a615-d230ef3a0101 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:53.047854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321696d5-c86d-4b40-a615-d230ef3a0101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.085470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-13a5a81f-73d6-4d0c-a5a0-cfac471f43e5 None None] MFA Rules not processed for user `c305307caa7e46fb913b9a71478bb5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:53.087724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-13a5a81f-73d6-4d0c-a5a0-cfac471f43e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:53.144163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-321696d5-c86d-4b40-a615-d230ef3a0101 None None] MFA Rules not processed for user `ee487075fd944918b01a158eeef811dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:53.147593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-321696d5-c86d-4b40-a615-d230ef3a0101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:53.252823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1067/2142] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:53.254284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5a81f-73d6-4d0c-a5a0-cfac471f43e5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:53.283255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1076/2143] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:53.284642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321696d5-c86d-4b40-a615-d230ef3a0101 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:53.349946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:53.419279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321696d5-c86d-4b40-a615-d230ef3a0101 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:53.425116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c48bddd-2de2-4f60-9d1e-a9e94d9b4c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:53.430586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48bddd-2de2-4f60-9d1e-a9e94d9b4c45 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:53.431124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48bddd-2de2-4f60-9d1e-a9e94d9b4c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:53.431458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c48bddd-2de2-4f60-9d1e-a9e94d9b4c45 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.432190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48bddd-2de2-4f60-9d1e-a9e94d9b4c45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:53.434414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c48bddd-2de2-4f60-9d1e-a9e94d9b4c45 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:53.453409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5a81f-73d6-4d0c-a5a0-cfac471f43e5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:53.458464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14204d7-f5a5-4053-962f-34e325d41c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:53.459480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14204d7-f5a5-4053-962f-34e325d41c0f admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:53.462044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14204d7-f5a5-4053-962f-34e325d41c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:53.462044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14204d7-f5a5-4053-962f-34e325d41c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.462044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14204d7-f5a5-4053-962f-34e325d41c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:53.462995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14204d7-f5a5-4053-962f-34e325d41c0f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:53.534348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1077/2144] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:53 2022] POST /identity/v3/projects => generated 387 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:53.535668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bf274-77de-4fd1-9870-b871e8e9dbf4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:53.536101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bf274-77de-4fd1-9870-b871e8e9dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:53.536450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bf274-77de-4fd1-9870-b871e8e9dbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.564417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1068/2145] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:53 2022] POST /identity/v3/projects => generated 385 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:53.565288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14204d7-f5a5-4053-962f-34e325d41c0f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:53.581733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3d9bf274-77de-4fd1-9870-b871e8e9dbf4 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:53.583063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d9bf274-77de-4fd1-9870-b871e8e9dbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:53.668943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1078/2146] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:55:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:53.669084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14204d7-f5a5-4053-962f-34e325d41c0f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:53.669916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bf274-77de-4fd1-9870-b871e8e9dbf4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:53.675648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:53.676546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:53.676924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:53.677312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.678110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:53.688022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:53.688219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bf274-77de-4fd1-9870-b871e8e9dbf4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:53.692287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d25ef-6f9f-4949-9649-b2c1143d1a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:53.693510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d25ef-6f9f-4949-9649-b2c1143d1a36 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:53.693989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d25ef-6f9f-4949-9649-b2c1143d1a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:53.694419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d25ef-6f9f-4949-9649-b2c1143d1a36 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.695259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d25ef-6f9f-4949-9649-b2c1143d1a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:53.698493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d25ef-6f9f-4949-9649-b2c1143d1a36 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:53.804318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1069/2147] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:53 2022] POST /identity/v3/users => generated 300 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:53.805350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:53.808537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:53.831254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1079/2148] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:53 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:53.832463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d25ef-6f9f-4949-9649-b2c1143d1a36 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:53.851582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d25ef-6f9f-4949-9649-b2c1143d1a36 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:53.869118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d5335-f897-447d-83d8-ca1ff66dbae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:53.870025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d5335-f897-447d-83d8-ca1ff66dbae1 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:53.870761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d5335-f897-447d-83d8-ca1ff66dbae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:53.871227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d5335-f897-447d-83d8-ca1ff66dbae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.876721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d5335-f897-447d-83d8-ca1ff66dbae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:53.876721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d5335-f897-447d-83d8-ca1ff66dbae1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:53.930065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1080/2149] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:53.931594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d5335-f897-447d-83d8-ca1ff66dbae1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:53.966286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d5335-f897-447d-83d8-ca1ff66dbae1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:53.975993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db525a4-b959-432b-b00a-e25b16e5030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:53.976928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db525a4-b959-432b-b00a-e25b16e5030f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:53.977115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db525a4-b959-432b-b00a-e25b16e5030f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:53.978261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db525a4-b959-432b-b00a-e25b16e5030f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:53.979172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db525a4-b959-432b-b00a-e25b16e5030f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:53.981425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db525a4-b959-432b-b00a-e25b16e5030f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:53.996669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1081/2150] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:53.998645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6dade4-9300-46cf-8c50-4d946a11a6c9 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:54.001959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db525a4-b959-432b-b00a-e25b16e5030f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:54.004615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c185a97-40c8-4a5c-a9a2-c90099be382a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:54.005447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c185a97-40c8-4a5c-a9a2-c90099be382a service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.008576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c185a97-40c8-4a5c-a9a2-c90099be382a service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.008576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c185a97-40c8-4a5c-a9a2-c90099be382a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.013613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db525a4-b959-432b-b00a-e25b16e5030f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:54.015832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c185a97-40c8-4a5c-a9a2-c90099be382a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:54.018359 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c185a97-40c8-4a5c-a9a2-c90099be382a service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:54.018852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7116bcc-98e1-47a0-b461-1594a2707b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:54.021265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7116bcc-98e1-47a0-b461-1594a2707b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.021564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7116bcc-98e1-47a0-b461-1594a2707b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.021855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7116bcc-98e1-47a0-b461-1594a2707b14 admin admin] PATH_INFO: `/v3/projects/6c66ebff07ef4c8ea2b95bd2bdc7c4ad/users/80be1f4f960f40a9b1c48d1e6b1722ce/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.028000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1070/2151] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:55:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:55:54.029100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c185a97-40c8-4a5c-a9a2-c90099be382a service swift] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:54.075369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c185a97-40c8-4a5c-a9a2-c90099be382a service swift] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:54.083129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7116bcc-98e1-47a0-b461-1594a2707b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c66ebff07ef4c8ea2b95bd2bdc7c4ad, user_id=80be1f4f960f40a9b1c48d1e6b1722ce, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:54.088601 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7116bcc-98e1-47a0-b461-1594a2707b14 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:54.089295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d458d1-1339-49e7-b81c-c10dd592fa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:54.090110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d458d1-1339-49e7-b81c-c10dd592fa38 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.093660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d458d1-1339-49e7-b81c-c10dd592fa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.094203 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d458d1-1339-49e7-b81c-c10dd592fa38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.095089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d458d1-1339-49e7-b81c-c10dd592fa38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:54.098514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d458d1-1339-49e7-b81c-c10dd592fa38 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:54.126247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1071/2152] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:54.127420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d458d1-1339-49e7-b81c-c10dd592fa38 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:54.152720 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d458d1-1339-49e7-b81c-c10dd592fa38 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:54.187779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83a395-a568-421c-891a-0543914dcaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:54.192034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83a395-a568-421c-891a-0543914dcaf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.193562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83a395-a568-421c-891a-0543914dcaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.194555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83a395-a568-421c-891a-0543914dcaf5 admin admin] PATH_INFO: `/v3/projects/52d3766893474c2ba9a7663cd400ae5c/users/da8c90520d7f47fdb687ded242f4e498/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.207518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1082/2153] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:54 2022] PUT /identity/v3/projects/6c66ebff07ef4c8ea2b95bd2bdc7c4ad/users/80be1f4f960f40a9b1c48d1e6b1722ce/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:54.212656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7116bcc-98e1-47a0-b461-1594a2707b14 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:54.237274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83a395-a568-421c-891a-0543914dcaf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d3766893474c2ba9a7663cd400ae5c, user_id=da8c90520d7f47fdb687ded242f4e498, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:54.240033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83a395-a568-421c-891a-0543914dcaf5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:54.386369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:54.394819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1072/2154] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:54 2022] PUT /identity/v3/projects/52d3766893474c2ba9a7663cd400ae5c/users/da8c90520d7f47fdb687ded242f4e498/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:54.409064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38ad1c-8580-4992-96d7-c7d83d8a7dc3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.410250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38ad1c-8580-4992-96d7-c7d83d8a7dc3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.410250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38ad1c-8580-4992-96d7-c7d83d8a7dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.446384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7116bcc-98e1-47a0-b461-1594a2707b14 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:54.491852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5f38ad1c-8580-4992-96d7-c7d83d8a7dc3 None None] MFA Rules not processed for user `da8c90520d7f47fdb687ded242f4e498`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:54.493043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5f38ad1c-8580-4992-96d7-c7d83d8a7dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:54.639180 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1073/2155] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:54.655175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fde7b8-9380-4ed2-a1ed-c3557ff9ff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:54.655918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fde7b8-9380-4ed2-a1ed-c3557ff9ff1e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.656211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fde7b8-9380-4ed2-a1ed-c3557ff9ff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.656493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fde7b8-9380-4ed2-a1ed-c3557ff9ff1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.657245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fde7b8-9380-4ed2-a1ed-c3557ff9ff1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:54.659601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38ad1c-8580-4992-96d7-c7d83d8a7dc3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:54.661186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fde7b8-9380-4ed2-a1ed-c3557ff9ff1e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:54.684261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38ad1c-8580-4992-96d7-c7d83d8a7dc3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:54.702706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8393c44-636f-46ac-8d39-8b99dae9034f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:54.703522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8393c44-636f-46ac-8d39-8b99dae9034f admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.703926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8393c44-636f-46ac-8d39-8b99dae9034f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.704387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8393c44-636f-46ac-8d39-8b99dae9034f admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.705463 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8393c44-636f-46ac-8d39-8b99dae9034f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:54.706992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1083/2156] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:54.708123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8393c44-636f-46ac-8d39-8b99dae9034f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:54.708336 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fde7b8-9380-4ed2-a1ed-c3557ff9ff1e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:54.741559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fde7b8-9380-4ed2-a1ed-c3557ff9ff1e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:54.871345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ea35d-a66d-45b1-84d6-30f22a969584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:54.871345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ea35d-a66d-45b1-84d6-30f22a969584 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.871345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ea35d-a66d-45b1-84d6-30f22a969584 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.871345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ea35d-a66d-45b1-84d6-30f22a969584 admin admin] PATH_INFO: `/v3/projects/6c66ebff07ef4c8ea2b95bd2bdc7c4ad/users/80be1f4f960f40a9b1c48d1e6b1722ce/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.872262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ea35d-a66d-45b1-84d6-30f22a969584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c66ebff07ef4c8ea2b95bd2bdc7c4ad, user_id=80be1f4f960f40a9b1c48d1e6b1722ce, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:54.872262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ea35d-a66d-45b1-84d6-30f22a969584 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:54.872262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1084/2157] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:54 2022] PUT /identity/v3/projects/6c66ebff07ef4c8ea2b95bd2bdc7c4ad/users/80be1f4f960f40a9b1c48d1e6b1722ce/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:54.872262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa621d0-0c86-4bb0-9f1c-e01d811391a7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:54.872262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa621d0-0c86-4bb0-9f1c-e01d811391a7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:54.872262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa621d0-0c86-4bb0-9f1c-e01d811391a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:54.882705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7aa621d0-0c86-4bb0-9f1c-e01d811391a7 None None] MFA Rules not processed for user `80be1f4f960f40a9b1c48d1e6b1722ce`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:54.889657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7aa621d0-0c86-4bb0-9f1c-e01d811391a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:55.052557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1085/2158] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:55:55.066556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa621d0-0c86-4bb0-9f1c-e01d811391a7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:55.068394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7aa621d0-0c86-4bb0-9f1c-e01d811391a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:55.146644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa621d0-0c86-4bb0-9f1c-e01d811391a7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:55.150845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce7b4b-7e8f-40d0-99bd-cf46333a25fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:55.152769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce7b4b-7e8f-40d0-99bd-cf46333a25fb service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:55.153175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce7b4b-7e8f-40d0-99bd-cf46333a25fb service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:55.153564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce7b4b-7e8f-40d0-99bd-cf46333a25fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:55.165989 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce7b4b-7e8f-40d0-99bd-cf46333a25fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:55.170723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce7b4b-7e8f-40d0-99bd-cf46333a25fb service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:55.185030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1086/2159] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:55.290616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1074/2160] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:55:54 2022] POST /identity/v3/projects => generated 385 bytes in 630 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:55.291853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce7b4b-7e8f-40d0-99bd-cf46333a25fb service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:55.492143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce7b4b-7e8f-40d0-99bd-cf46333a25fb service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:55.765913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d342a3-5fb3-448f-af23-cb6bbdef1ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:55.766631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d342a3-5fb3-448f-af23-cb6bbdef1ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:55.766916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d342a3-5fb3-448f-af23-cb6bbdef1ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:55.767220 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d342a3-5fb3-448f-af23-cb6bbdef1ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:55.768421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d342a3-5fb3-448f-af23-cb6bbdef1ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:55.770326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d342a3-5fb3-448f-af23-cb6bbdef1ac5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:56.343917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1087/2161] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:55:55 2022] POST /identity/v3/users => generated 299 bytes in 1028 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:55:56.344670 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8393c44-636f-46ac-8d39-8b99dae9034f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:56.344670 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8393c44-636f-46ac-8d39-8b99dae9034f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:56.355844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b92aa-808a-4a42-9447-023e1d822ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:56.358538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b92aa-808a-4a42-9447-023e1d822ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:56.358795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b92aa-808a-4a42-9447-023e1d822ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:56.366465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b92aa-808a-4a42-9447-023e1d822ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:56.367538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b92aa-808a-4a42-9447-023e1d822ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:56.369377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b92aa-808a-4a42-9447-023e1d822ed7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:56.415690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1075/2162] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:56.422638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d342a3-5fb3-448f-af23-cb6bbdef1ac5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:56.436432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d342a3-5fb3-448f-af23-cb6bbdef1ac5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:56.452963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b67e3-293a-4eaf-a3dd-db7b062b390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:56.452963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b67e3-293a-4eaf-a3dd-db7b062b390a admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:56.452963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b67e3-293a-4eaf-a3dd-db7b062b390a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:56.452963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b67e3-293a-4eaf-a3dd-db7b062b390a admin admin] PATH_INFO: `/v3/projects/701c4ccb2976422ebbf4487c2f2695b4/users/9f6081f0fbf94f1dab3298ba71749678/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:56.476252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b67e3-293a-4eaf-a3dd-db7b062b390a admin admin] RBAC: Authorizing `identity:create_grant(project_id=701c4ccb2976422ebbf4487c2f2695b4, user_id=9f6081f0fbf94f1dab3298ba71749678, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:56.481302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b67e3-293a-4eaf-a3dd-db7b062b390a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:56.670962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1088/2163] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:56 2022] PUT /identity/v3/projects/701c4ccb2976422ebbf4487c2f2695b4/users/9f6081f0fbf94f1dab3298ba71749678/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:56.676784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b92aa-808a-4a42-9447-023e1d822ed7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:56.693281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b92aa-808a-4a42-9447-023e1d822ed7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:56.853551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:56.862615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856c737-898d-4774-9e0f-9f2305cca4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:56.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856c737-898d-4774-9e0f-9f2305cca4ce admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:56.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856c737-898d-4774-9e0f-9f2305cca4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:56.868890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856c737-898d-4774-9e0f-9f2305cca4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:56.869933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856c737-898d-4774-9e0f-9f2305cca4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:56.876376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856c737-898d-4774-9e0f-9f2305cca4ce admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:56.898395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1076/2164] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:55:56.899733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b67e3-293a-4eaf-a3dd-db7b062b390a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:56.914099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b67e3-293a-4eaf-a3dd-db7b062b390a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:56.952497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c853e-da22-4200-81da-13752d8ae710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:56.956283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c853e-da22-4200-81da-13752d8ae710 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:56.956865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c853e-da22-4200-81da-13752d8ae710 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:56.957388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c853e-da22-4200-81da-13752d8ae710 admin admin] PATH_INFO: `/v3/projects/701c4ccb2976422ebbf4487c2f2695b4/users/9f6081f0fbf94f1dab3298ba71749678/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:56.967104 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c853e-da22-4200-81da-13752d8ae710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=701c4ccb2976422ebbf4487c2f2695b4, user_id=9f6081f0fbf94f1dab3298ba71749678, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:56.970354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c853e-da22-4200-81da-13752d8ae710 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:56.991428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1089/2165] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:55:56 2022] PUT /identity/v3/projects/701c4ccb2976422ebbf4487c2f2695b4/users/9f6081f0fbf94f1dab3298ba71749678/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:56.996767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163c9b-c42b-4adc-bbf2-4a6399cf195b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:57.002052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163c9b-c42b-4adc-bbf2-4a6399cf195b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:57.002052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163c9b-c42b-4adc-bbf2-4a6399cf195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:57.060892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-89163c9b-c42b-4adc-bbf2-4a6399cf195b None None] MFA Rules not processed for user `9f6081f0fbf94f1dab3298ba71749678`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:57.066744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-89163c9b-c42b-4adc-bbf2-4a6399cf195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:57.380304 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1077/2166] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:57.388039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c853e-da22-4200-81da-13752d8ae710 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:57.435496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89163c9b-c42b-4adc-bbf2-4a6399cf195b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:57.437083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-89163c9b-c42b-4adc-bbf2-4a6399cf195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:57.648240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89163c9b-c42b-4adc-bbf2-4a6399cf195b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:57.669605 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f014c-c979-40a1-944d-93e50749bd3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:57.670303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f014c-c979-40a1-944d-93e50749bd3d service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:57.670587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f014c-c979-40a1-944d-93e50749bd3d service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:57.670877 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f014c-c979-40a1-944d-93e50749bd3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:57.720731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c853e-da22-4200-81da-13752d8ae710 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:57.725699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:57.726696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:57.727155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:57.727636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:57.843070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f014c-c979-40a1-944d-93e50749bd3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:57.850374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f014c-c979-40a1-944d-93e50749bd3d service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:57.860856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1078/2167] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:55:57.928476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:57.931998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:57.955709 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1090/2168] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:57.977147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:58.862331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526f1a3-2a83-4435-b9f9-187da562f5d7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:58.862903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526f1a3-2a83-4435-b9f9-187da562f5d7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:58.863683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526f1a3-2a83-4435-b9f9-187da562f5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:58.957127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7526f1a3-2a83-4435-b9f9-187da562f5d7 None None] MFA Rules not processed for user `1157b91a306540dd98a072b1ea4941b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:55:58.958371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7526f1a3-2a83-4435-b9f9-187da562f5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:59.227855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1079/2169] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:55:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:55:59.244095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:59.247747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:59.390619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:59.443083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02516e-2dc5-48c5-91e5-41f861238a95 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:59.455832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223e683-e5fd-4208-8b72-2e1b87c66a8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:59.457304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e683-e5fd-4208-8b72-2e1b87c66a8e service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:59.457902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e683-e5fd-4208-8b72-2e1b87c66a8e service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:59.458197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223e683-e5fd-4208-8b72-2e1b87c66a8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:59.471185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223e683-e5fd-4208-8b72-2e1b87c66a8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:59.472967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223e683-e5fd-4208-8b72-2e1b87c66a8e service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:59.487801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1091/2170] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:01.878972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:02.944404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526f1a3-2a83-4435-b9f9-187da562f5d7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:02.959637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526f1a3-2a83-4435-b9f9-187da562f5d7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:02.968813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c7a61-8ce4-40ad-ab6e-024f94adaad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:02.974014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c7a61-8ce4-40ad-ab6e-024f94adaad1 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:02.974014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c7a61-8ce4-40ad-ab6e-024f94adaad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:02.974014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c7a61-8ce4-40ad-ab6e-024f94adaad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:03.122457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c7a61-8ce4-40ad-ab6e-024f94adaad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:03.124008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c7a61-8ce4-40ad-ab6e-024f94adaad1 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:03.146873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1080/2171] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:03.700947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d5c7c-77a8-4e10-9bed-5f483cd38bbf None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:03.701973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d5c7c-77a8-4e10-9bed-5f483cd38bbf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:03.702410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d5c7c-77a8-4e10-9bed-5f483cd38bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:03.748463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-da2d5c7c-77a8-4e10-9bed-5f483cd38bbf None None] MFA Rules not processed for user `1157b91a306540dd98a072b1ea4941b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:03.749978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-da2d5c7c-77a8-4e10-9bed-5f483cd38bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:03.773807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1092/2172] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:04.090977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:04.970422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:05.617782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c7a61-8ce4-40ad-ab6e-024f94adaad1 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:05.631175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c7a61-8ce4-40ad-ab6e-024f94adaad1 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:05.640869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951731d-32f6-4746-bf2c-e4ff78485b24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:05.643167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951731d-32f6-4746-bf2c-e4ff78485b24 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:05.643521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951731d-32f6-4746-bf2c-e4ff78485b24 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:05.644125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951731d-32f6-4746-bf2c-e4ff78485b24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:05.665156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951731d-32f6-4746-bf2c-e4ff78485b24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:05.667267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951731d-32f6-4746-bf2c-e4ff78485b24 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:05.680608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1081/2173] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:07.025376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5726ce-7cd1-43ea-8ed4-c75813cba065 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:07.026010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5726ce-7cd1-43ea-8ed4-c75813cba065 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:07.026010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5726ce-7cd1-43ea-8ed4-c75813cba065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:07.044919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2f5726ce-7cd1-43ea-8ed4-c75813cba065 None None] MFA Rules not processed for user `9005cb23317a4a05b58e99284824135e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:07.046173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2f5726ce-7cd1-43ea-8ed4-c75813cba065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:07.109778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1093/2174] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:07.528309 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:07.678423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951731d-32f6-4746-bf2c-e4ff78485b24 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:07.696812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951731d-32f6-4746-bf2c-e4ff78485b24 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:07.707118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2e1bd-9305-4888-adba-8acb40d1d078 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:07.709862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2e1bd-9305-4888-adba-8acb40d1d078 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:07.710216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2e1bd-9305-4888-adba-8acb40d1d078 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:07.711068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2e1bd-9305-4888-adba-8acb40d1d078 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:07.830964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2e1bd-9305-4888-adba-8acb40d1d078 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:07.832829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2e1bd-9305-4888-adba-8acb40d1d078 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:07.844892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1082/2175] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:07.941001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:08.204706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:09.499125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:09.584057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:11.753204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0f2fa-f458-4e9d-a133-cac00e4fe073 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:11.753747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0f2fa-f458-4e9d-a133-cac00e4fe073 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:11.753747 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0f2fa-f458-4e9d-a133-cac00e4fe073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:11.776522 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a7f0f2fa-f458-4e9d-a133-cac00e4fe073 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:11.777767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a7f0f2fa-f458-4e9d-a133-cac00e4fe073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:11.834921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1094/2176] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:56:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:11.861233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2e1bd-9305-4888-adba-8acb40d1d078 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:11.863450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877e717-0951-4c84-a439-2814a7e5d422 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:11.863984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877e717-0951-4c84-a439-2814a7e5d422 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:11.864473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877e717-0951-4c84-a439-2814a7e5d422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:11.884847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3877e717-0951-4c84-a439-2814a7e5d422 None None] MFA Rules not processed for user `bff41a6af0ea495a856fd5603e810d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:11.886088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3877e717-0951-4c84-a439-2814a7e5d422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:11.924412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2e1bd-9305-4888-adba-8acb40d1d078 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:11.929478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831dac8-e61b-4329-bd57-8e13d03c71b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:11.930185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831dac8-e61b-4329-bd57-8e13d03c71b2 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:11.930592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831dac8-e61b-4329-bd57-8e13d03c71b2 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:11.930874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831dac8-e61b-4329-bd57-8e13d03c71b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:11.939046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:11.939312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831dac8-e61b-4329-bd57-8e13d03c71b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:11.940977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831dac8-e61b-4329-bd57-8e13d03c71b2 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:11.943588 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1095/2177] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:11.949715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1083/2178] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:56:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:56:12.236113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:13.663671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:14.573173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:15.127645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877e717-0951-4c84-a439-2814a7e5d422 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:15.305263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877e717-0951-4c84-a439-2814a7e5d422 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:15.318884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:15.319818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:15.320103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:15.320388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] PATH_INFO: `/v3/users/bdda1d2d0cbb46dfb50c220ffcaecdaa` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:15.324934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdda1d2d0cbb46dfb50c220ffcaecdaa)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:15.327749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:15.653231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdda1d2d0cbb46dfb50c220ffcaecdaa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:15.680888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdda1d2d0cbb46dfb50c220ffcaecdaa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:15.682496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdda1d2d0cbb46dfb50c220ffcaecdaa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:15.683112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdda1d2d0cbb46dfb50c220ffcaecdaa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:15.691945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdda1d2d0cbb46dfb50c220ffcaecdaa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:15.803039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1096/2179] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:15 2022] DELETE /identity/v3/users/bdda1d2d0cbb46dfb50c220ffcaecdaa => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:15.809398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57818055-3376-45f0-9314-bb9843b34cc8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:15.809964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57818055-3376-45f0-9314-bb9843b34cc8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:15.810047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57818055-3376-45f0-9314-bb9843b34cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:15.901129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-57818055-3376-45f0-9314-bb9843b34cc8 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:15.906416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-57818055-3376-45f0-9314-bb9843b34cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:16.080945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1084/2180] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:16.097825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:16.102389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:16.197577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec240e-2398-45f3-9591-1e447ea0521a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:16.202407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd3b51-10dd-4c6a-a6aa-c71502eead48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:16.203278 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd3b51-10dd-4c6a-a6aa-c71502eead48 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:16.203845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd3b51-10dd-4c6a-a6aa-c71502eead48 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:16.204324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd3b51-10dd-4c6a-a6aa-c71502eead48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:16.219974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd3b51-10dd-4c6a-a6aa-c71502eead48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:16.222387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd3b51-10dd-4c6a-a6aa-c71502eead48 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:16.236901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1097/2181] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:16.868564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57818055-3376-45f0-9314-bb9843b34cc8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:16.870727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-57818055-3376-45f0-9314-bb9843b34cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:16.884893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57818055-3376-45f0-9314-bb9843b34cc8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:16.892748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:16.894068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:16.894767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:16.895276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] PATH_INFO: `/v3/users/c305307caa7e46fb913b9a71478bb5a3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:16.897633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c305307caa7e46fb913b9a71478bb5a3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:16.908488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:17.143872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c305307caa7e46fb913b9a71478bb5a3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.171497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c305307caa7e46fb913b9a71478bb5a3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.173779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c305307caa7e46fb913b9a71478bb5a3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.174420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c305307caa7e46fb913b9a71478bb5a3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.219438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c305307caa7e46fb913b9a71478bb5a3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.380865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44315f-fcbd-495f-9e21-791a4d409e16 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:17.381416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44315f-fcbd-495f-9e21-791a4d409e16 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:17.381416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44315f-fcbd-495f-9e21-791a4d409e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:17.388111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1085/2182] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:16 2022] DELETE /identity/v3/users/c305307caa7e46fb913b9a71478bb5a3 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:17.420547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0b44315f-fcbd-495f-9e21-791a4d409e16 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:17.425615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0b44315f-fcbd-495f-9e21-791a4d409e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:17.463561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:17.579883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1098/2183] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:17.587139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:17.589344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:17.604419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44315f-fcbd-495f-9e21-791a4d409e16 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:17.620682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0b44315f-fcbd-495f-9e21-791a4d409e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:17.621006 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003c3e7-335f-44ce-bf6c-0c90c142b0bf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:17.636028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:17.636957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:17.637356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:17.637541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] PATH_INFO: `/v3/projects/d0ba8f444fc247f98273872bc7dab144` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:17.644069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ba8f444fc247f98273872bc7dab144)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:17.646083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:17.726772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44315f-fcbd-495f-9e21-791a4d409e16 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:17.733494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c86b55-74bb-42ce-a912-2365acb6ff9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:17.735078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86b55-74bb-42ce-a912-2365acb6ff9a service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:17.735688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86b55-74bb-42ce-a912-2365acb6ff9a service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:17.736142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c86b55-74bb-42ce-a912-2365acb6ff9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:17.749688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c86b55-74bb-42ce-a912-2365acb6ff9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:17.752174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c86b55-74bb-42ce-a912-2365acb6ff9a service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:17.768830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1099/2184] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:17.818166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dfb27-0c71-47f2-8cc1-900818feca2e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:17.818464 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dfb27-0c71-47f2-8cc1-900818feca2e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:17.819067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dfb27-0c71-47f2-8cc1-900818feca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:17.843098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:17.851752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-355dfb27-0c71-47f2-8cc1-900818feca2e None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:17.853794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ba8f444fc247f98273872bc7dab144', 'request_id': 'req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.854722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-355dfb27-0c71-47f2-8cc1-900818feca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:17.860734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ba8f444fc247f98273872bc7dab144', 'request_id': 'req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.867376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ba8f444fc247f98273872bc7dab144', 'request_id': 'req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.948849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] The token cache is being invalidate because project d0ba8f444fc247f98273872bc7dab144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:17.949600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.951028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:17.952741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1100/2185] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:17.956416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1086/2186] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:17 2022] DELETE /identity/v3/projects/d0ba8f444fc247f98273872bc7dab144 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:17.960902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dfb27-0c71-47f2-8cc1-900818feca2e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:17.982198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:17.983732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:18.168050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dfb27-0c71-47f2-8cc1-900818feca2e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:18.184015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:18.184921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:18.185213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:18.185487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] PATH_INFO: `/v3/users/ee487075fd944918b01a158eeef811dc` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:18.187427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee487075fd944918b01a158eeef811dc)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:18.195237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:18.253827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e3948-fb69-43e5-a4c4-14a45b6e7794 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:18.267494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:18.268189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:18.268655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:18.269842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:18.272684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:18.288074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:18.294952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:18.305482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1087/2187] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 15:56:18.499584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee487075fd944918b01a158eeef811dc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:18.531619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee487075fd944918b01a158eeef811dc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:18.532639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee487075fd944918b01a158eeef811dc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:18.533423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee487075fd944918b01a158eeef811dc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:18.546877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee487075fd944918b01a158eeef811dc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:18.663833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1101/2188] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:17 2022] DELETE /identity/v3/users/ee487075fd944918b01a158eeef811dc => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:18.766836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:18.767017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:18.800861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75de516-7864-430b-ade3-718ca23befdc service swift] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:18.972996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:18.974683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:18.976087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:18.987518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] PATH_INFO: `/v3/projects/febe1f57973d4c5cbd2cb66b597e5b9f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:18.990342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=febe1f57973d4c5cbd2cb66b597e5b9f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:18.996542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:19.246658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febe1f57973d4c5cbd2cb66b597e5b9f', 'request_id': 'req-fa699d01-1510-4c5b-9749-ab90525f3809'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.256487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febe1f57973d4c5cbd2cb66b597e5b9f', 'request_id': 'req-fa699d01-1510-4c5b-9749-ab90525f3809'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.272331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febe1f57973d4c5cbd2cb66b597e5b9f', 'request_id': 'req-fa699d01-1510-4c5b-9749-ab90525f3809'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.339908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:19.345431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:19.363139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1df990-d144-4f31-ba28-fdcffa711a41 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:19.386733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] The token cache is being invalidate because project febe1f57973d4c5cbd2cb66b597e5b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:19.387035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.387777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.393160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1088/2189] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:18 2022] DELETE /identity/v3/projects/febe1f57973d4c5cbd2cb66b597e5b9f => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:19.394512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:19.396576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:19.478329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:19.478329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:19.478329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:19.478329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] PATH_INFO: `/v3/projects/fbd05f8be6b0477cbb1ff0bc8b6e12c5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:19.479953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd05f8be6b0477cbb1ff0bc8b6e12c5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:19.482489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:19.540870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa699d01-1510-4c5b-9749-ab90525f3809 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:19.547177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:19.548431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:19.548961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:19.549457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] PATH_INFO: `/v3/users/da8c90520d7f47fdb687ded242f4e498` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:19.551026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da8c90520d7f47fdb687ded242f4e498)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:19.553270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:19.680976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd05f8be6b0477cbb1ff0bc8b6e12c5', 'request_id': 'req-2b872ed8-a2a4-437c-8b21-18337b8eca00'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.693765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd05f8be6b0477cbb1ff0bc8b6e12c5', 'request_id': 'req-2b872ed8-a2a4-437c-8b21-18337b8eca00'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.813067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da8c90520d7f47fdb687ded242f4e498'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.818619 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd05f8be6b0477cbb1ff0bc8b6e12c5', 'request_id': 'req-2b872ed8-a2a4-437c-8b21-18337b8eca00'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.833529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da8c90520d7f47fdb687ded242f4e498'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.838123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da8c90520d7f47fdb687ded242f4e498'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.839434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da8c90520d7f47fdb687ded242f4e498'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.839910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] The token cache is being invalidate because project fbd05f8be6b0477cbb1ff0bc8b6e12c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:19.840389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.841531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.844420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1102/2190] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:19 2022] DELETE /identity/v3/projects/fbd05f8be6b0477cbb1ff0bc8b6e12c5 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:56:19.848613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:19.850436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:19.857129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da8c90520d7f47fdb687ded242f4e498'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:19.934874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b872ed8-a2a4-437c-8b21-18337b8eca00 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:19.940040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:19.940985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:19.941350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:19.941703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] PATH_INFO: `/v3/users/80be1f4f960f40a9b1c48d1e6b1722ce` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:19.942949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80be1f4f960f40a9b1c48d1e6b1722ce)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:19.944970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:19.996137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1089/2191] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:19 2022] DELETE /identity/v3/users/da8c90520d7f47fdb687ded242f4e498 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:20.203281 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80be1f4f960f40a9b1c48d1e6b1722ce'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:20.219731 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80be1f4f960f40a9b1c48d1e6b1722ce'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:20.222498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80be1f4f960f40a9b1c48d1e6b1722ce'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:20.222498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80be1f4f960f40a9b1c48d1e6b1722ce'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:20.232599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80be1f4f960f40a9b1c48d1e6b1722ce'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:20.322036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1103/2192] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:19 2022] DELETE /identity/v3/users/80be1f4f960f40a9b1c48d1e6b1722ce => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:20.547508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:20.698061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:20.698061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:20.833272 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:20.908258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e06161-ac02-4681-8070-225ecc95e1c1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:20.973246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:20.974906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:21.020699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:21.021593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:21.021961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:21.022251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] PATH_INFO: `/v3/projects/52d3766893474c2ba9a7663cd400ae5c` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:21.035341 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d3766893474c2ba9a7663cd400ae5c)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:21.061515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:21.134502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05b5b1-8a32-4311-bc4d-e004feec3177 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:21.142761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:21.144092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:21.144504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:21.144946 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] PATH_INFO: `/v3/projects/6c66ebff07ef4c8ea2b95bd2bdc7c4ad` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:21.146181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c66ebff07ef4c8ea2b95bd2bdc7c4ad)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:21.148237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:21.215904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d3766893474c2ba9a7663cd400ae5c', 'request_id': 'req-08c9758d-53c1-417c-96bf-cd2bfbc656b1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.231739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d3766893474c2ba9a7663cd400ae5c', 'request_id': 'req-08c9758d-53c1-417c-96bf-cd2bfbc656b1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.248015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d3766893474c2ba9a7663cd400ae5c', 'request_id': 'req-08c9758d-53c1-417c-96bf-cd2bfbc656b1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.359348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c66ebff07ef4c8ea2b95bd2bdc7c4ad', 'request_id': 'req-1d9b7ac1-8341-4883-9700-9288821841c9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.364383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] The token cache is being invalidate because project 52d3766893474c2ba9a7663cd400ae5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:21.364690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.367662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c66ebff07ef4c8ea2b95bd2bdc7c4ad', 'request_id': 'req-1d9b7ac1-8341-4883-9700-9288821841c9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.369297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.377978 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1090/2193] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:20 2022] DELETE /identity/v3/projects/52d3766893474c2ba9a7663cd400ae5c => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:21.380019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:21.381865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:21.495744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c66ebff07ef4c8ea2b95bd2bdc7c4ad', 'request_id': 'req-1d9b7ac1-8341-4883-9700-9288821841c9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.520095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] The token cache is being invalidate because project 6c66ebff07ef4c8ea2b95bd2bdc7c4ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:21.522331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.524167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d9b7ac1-8341-4883-9700-9288821841c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.528944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1104/2194] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:20 2022] DELETE /identity/v3/projects/6c66ebff07ef4c8ea2b95bd2bdc7c4ad => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:21.534887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8efb0f-7cc0-40c8-9d73-d37c6b6de765 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:21.535194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8efb0f-7cc0-40c8-9d73-d37c6b6de765 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:21.535482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8efb0f-7cc0-40c8-9d73-d37c6b6de765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:21.541599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9758d-53c1-417c-96bf-cd2bfbc656b1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:21.571920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ef8efb0f-7cc0-40c8-9d73-d37c6b6de765 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:21.573528 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ef8efb0f-7cc0-40c8-9d73-d37c6b6de765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:21.638955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:21.640264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:21.640561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:21.640835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] PATH_INFO: `/v3/users/9f6081f0fbf94f1dab3298ba71749678` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:21.642584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f6081f0fbf94f1dab3298ba71749678)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:21.646401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:21.661739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1105/2195] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:21.677657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41875bb-7d3b-4506-890f-9c6580090fab None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:21.681831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41875bb-7d3b-4506-890f-9c6580090fab None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:21.681831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41875bb-7d3b-4506-890f-9c6580090fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:21.712880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e41875bb-7d3b-4506-890f-9c6580090fab None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:21.714718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e41875bb-7d3b-4506-890f-9c6580090fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:21.732265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1106/2196] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:21.733933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41875bb-7d3b-4506-890f-9c6580090fab None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:21.748656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41875bb-7d3b-4506-890f-9c6580090fab None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:21.754129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:21.754928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:21.755580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:21.755714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:21.756641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:56:21.756959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:21.758651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:21.792076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1107/2197] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:56:21.801651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:21.821745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d5f1c-4b7f-498d-92e6-4edb0031f2ab admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:21.829370 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:21.830115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:21.830704 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:21.834540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:21.835979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:21.840592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:21.953680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f6081f0fbf94f1dab3298ba71749678'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.976380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f6081f0fbf94f1dab3298ba71749678'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.977971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f6081f0fbf94f1dab3298ba71749678'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.978863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f6081f0fbf94f1dab3298ba71749678'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:21.996665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f6081f0fbf94f1dab3298ba71749678'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:22.017177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1108/2198] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:21 2022] POST /identity/v3/projects => generated 393 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:22.019697 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:22.024951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:22.093258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1091/2199] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:21 2022] DELETE /identity/v3/users/9f6081f0fbf94f1dab3298ba71749678 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:56:22.174247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b1ae6-8a67-4917-a030-bd49a433391e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:22.314211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc300f-ba02-412b-93b0-652b2cf34495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:22.314874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc300f-ba02-412b-93b0-652b2cf34495 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:22.315167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc300f-ba02-412b-93b0-652b2cf34495 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:22.315473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc300f-ba02-412b-93b0-652b2cf34495 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:22.316178 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc300f-ba02-412b-93b0-652b2cf34495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:22.318139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc300f-ba02-412b-93b0-652b2cf34495 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:22.535442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1109/2200] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:22 2022] POST /identity/v3/users => generated 297 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:22.539713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:22.554905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb65fe3-fe1c-426e-ae75-8105271e0b79 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:22.567895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91cf4-dbb1-4698-9f1c-a185b25df830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:22.568798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91cf4-dbb1-4698-9f1c-a185b25df830 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:22.569167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91cf4-dbb1-4698-9f1c-a185b25df830 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:22.569521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91cf4-dbb1-4698-9f1c-a185b25df830 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:22.570330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91cf4-dbb1-4698-9f1c-a185b25df830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:22.573744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91cf4-dbb1-4698-9f1c-a185b25df830 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:22.600056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1092/2201] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:22.601356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc300f-ba02-412b-93b0-652b2cf34495 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:22.620112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc300f-ba02-412b-93b0-652b2cf34495 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:22.638784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d64d6-a013-47f4-8ba2-d223307b8286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:22.639505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d64d6-a013-47f4-8ba2-d223307b8286 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:22.639800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d64d6-a013-47f4-8ba2-d223307b8286 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:22.640084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d64d6-a013-47f4-8ba2-d223307b8286 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:22.640825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d64d6-a013-47f4-8ba2-d223307b8286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:22.643038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d64d6-a013-47f4-8ba2-d223307b8286 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:22.681474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91cf4-dbb1-4698-9f1c-a185b25df830 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:22.685965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1110/2202] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:22.699482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91cf4-dbb1-4698-9f1c-a185b25df830 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:22.705706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03816243-fc6e-4b1c-b0bb-a93ef90b0efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:22.706849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03816243-fc6e-4b1c-b0bb-a93ef90b0efd admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:22.707347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03816243-fc6e-4b1c-b0bb-a93ef90b0efd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:22.707777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03816243-fc6e-4b1c-b0bb-a93ef90b0efd admin admin] PATH_INFO: `/v3/projects/941ba64c0fc344e58f5b0311597e8c77/users/6ec33370d5634dc8a54eeeb2cf694211/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:22.746812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03816243-fc6e-4b1c-b0bb-a93ef90b0efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=941ba64c0fc344e58f5b0311597e8c77, user_id=6ec33370d5634dc8a54eeeb2cf694211, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:22.750022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03816243-fc6e-4b1c-b0bb-a93ef90b0efd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:22.757332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:22.839345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1093/2203] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:22 2022] PUT /identity/v3/projects/941ba64c0fc344e58f5b0311597e8c77/users/6ec33370d5634dc8a54eeeb2cf694211/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:22.849532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b364fe5-a3da-41a0-b99e-6c1c5d349f16 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:22.849910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b364fe5-a3da-41a0-b99e-6c1c5d349f16 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:22.850216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b364fe5-a3da-41a0-b99e-6c1c5d349f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:22.884137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3b364fe5-a3da-41a0-b99e-6c1c5d349f16 None None] MFA Rules not processed for user `6ec33370d5634dc8a54eeeb2cf694211`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:22.885994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3b364fe5-a3da-41a0-b99e-6c1c5d349f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:23.038264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1111/2204] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:23.044371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03816243-fc6e-4b1c-b0bb-a93ef90b0efd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:23.078789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b364fe5-a3da-41a0-b99e-6c1c5d349f16 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:23.081145 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3b364fe5-a3da-41a0-b99e-6c1c5d349f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:23.236478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b364fe5-a3da-41a0-b99e-6c1c5d349f16 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:23.242100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:23.243201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:23.243555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:23.244130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] PATH_INFO: `/v3/projects/701c4ccb2976422ebbf4487c2f2695b4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:23.246656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=701c4ccb2976422ebbf4487c2f2695b4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:23.272392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:23.324275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03816243-fc6e-4b1c-b0bb-a93ef90b0efd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:23.324275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d571575-dfb5-43d9-9ab6-b39e0bcd9b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:23.324275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d571575-dfb5-43d9-9ab6-b39e0bcd9b38 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:23.324275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d571575-dfb5-43d9-9ab6-b39e0bcd9b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:23.325121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d571575-dfb5-43d9-9ab6-b39e0bcd9b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:23.325121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d571575-dfb5-43d9-9ab6-b39e0bcd9b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:23.325276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d571575-dfb5-43d9-9ab6-b39e0bcd9b38 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:23.840517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:23.950181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '701c4ccb2976422ebbf4487c2f2695b4', 'request_id': 'req-ece1e903-80dd-41ff-87c3-ac5b83b996ca'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:23.957956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1094/2205] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:23 2022] POST /identity/v3/projects => generated 393 bytes in 911 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:23.958946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d571575-dfb5-43d9-9ab6-b39e0bcd9b38 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:23.968409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '701c4ccb2976422ebbf4487c2f2695b4', 'request_id': 'req-ece1e903-80dd-41ff-87c3-ac5b83b996ca'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:24.036930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '701c4ccb2976422ebbf4487c2f2695b4', 'request_id': 'req-ece1e903-80dd-41ff-87c3-ac5b83b996ca'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:24.070133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] The token cache is being invalidate because project 701c4ccb2976422ebbf4487c2f2695b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:24.077304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:24.077304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ece1e903-80dd-41ff-87c3-ac5b83b996ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:24.081772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1112/2206] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:23 2022] DELETE /identity/v3/projects/701c4ccb2976422ebbf4487c2f2695b4 => generated 0 bytes in 1003 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:24.086744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d06d7-efd9-43ca-9318-c1409128f1d5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.088554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d06d7-efd9-43ca-9318-c1409128f1d5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.088873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d06d7-efd9-43ca-9318-c1409128f1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.147370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d571575-dfb5-43d9-9ab6-b39e0bcd9b38 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.152302 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d96d06d7-efd9-43ca-9318-c1409128f1d5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:24.153685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d96d06d7-efd9-43ca-9318-c1409128f1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:24.161352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:24.163449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.169186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.169186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.169186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:24.169597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1113/2207] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:24.170691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:24.182651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362967b-c078-44d3-9bbf-a82fd8ebd191 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.183583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362967b-c078-44d3-9bbf-a82fd8ebd191 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.183882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362967b-c078-44d3-9bbf-a82fd8ebd191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.211067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0362967b-c078-44d3-9bbf-a82fd8ebd191 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:24.213775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0362967b-c078-44d3-9bbf-a82fd8ebd191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:24.226024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1114/2208] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:24.226966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362967b-c078-44d3-9bbf-a82fd8ebd191 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.238085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362967b-c078-44d3-9bbf-a82fd8ebd191 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.261358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:24.262603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.263949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.264461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.265611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:56:24.266197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:24.268461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:24.285990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1115/2209] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:56:24.291816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.298193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1095/2210] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:23 2022] POST /identity/v3/users => generated 303 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:24.304839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.307669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c608ba-94c7-4102-8f38-85814097b3d3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.309678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:24.314654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c08b0-8604-46d8-9ff4-1dd623015e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:24.316047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c08b0-8604-46d8-9ff4-1dd623015e92 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.316642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c08b0-8604-46d8-9ff4-1dd623015e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.317218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c08b0-8604-46d8-9ff4-1dd623015e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.318805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c08b0-8604-46d8-9ff4-1dd623015e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:24.321362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c08b0-8604-46d8-9ff4-1dd623015e92 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:24.326321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886859c6-c2bd-4df8-ba74-05913f737d1d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.333328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db6cf-030e-4bac-bcdd-7cdd223d6856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:24.346728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db6cf-030e-4bac-bcdd-7cdd223d6856 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.347426 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db6cf-030e-4bac-bcdd-7cdd223d6856 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.348048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799db6cf-030e-4bac-bcdd-7cdd223d6856 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.350699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799db6cf-030e-4bac-bcdd-7cdd223d6856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:24.354358 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799db6cf-030e-4bac-bcdd-7cdd223d6856 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:24.382625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1096/2211] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:24.390851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db6cf-030e-4bac-bcdd-7cdd223d6856 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.423209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799db6cf-030e-4bac-bcdd-7cdd223d6856 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.438530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763c58-80df-4a38-8902-4da42e603e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:24.440842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763c58-80df-4a38-8902-4da42e603e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.441251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763c58-80df-4a38-8902-4da42e603e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.442694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763c58-80df-4a38-8902-4da42e603e2b admin admin] PATH_INFO: `/v3/projects/ea78d9ee249e419a8041cfa69e06fb82/users/ce5708a27a274b71ad424a1cbe56c69c/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.446524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1116/2212] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:24 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:24.455205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c08b0-8604-46d8-9ff4-1dd623015e92 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.464187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763c58-80df-4a38-8902-4da42e603e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea78d9ee249e419a8041cfa69e06fb82, user_id=ce5708a27a274b71ad424a1cbe56c69c, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:24.468077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763c58-80df-4a38-8902-4da42e603e2b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:24.556762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1097/2213] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:24 2022] PUT /identity/v3/projects/ea78d9ee249e419a8041cfa69e06fb82/users/ce5708a27a274b71ad424a1cbe56c69c/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:24.561487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763c58-80df-4a38-8902-4da42e603e2b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.574463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c08b0-8604-46d8-9ff4-1dd623015e92 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.662887 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763c58-80df-4a38-8902-4da42e603e2b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.664816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2659116-1dcf-43fc-adb6-7a94232cf4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:24.665694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2659116-1dcf-43fc-adb6-7a94232cf4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.666071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2659116-1dcf-43fc-adb6-7a94232cf4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.666504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2659116-1dcf-43fc-adb6-7a94232cf4c8 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.667250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2659116-1dcf-43fc-adb6-7a94232cf4c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:24.671629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1014bdc-3c05-4378-b3e6-d7e1c633380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:24.671629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1014bdc-3c05-4378-b3e6-d7e1c633380b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.671862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2659116-1dcf-43fc-adb6-7a94232cf4c8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:24.671963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1014bdc-3c05-4378-b3e6-d7e1c633380b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.671963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1014bdc-3c05-4378-b3e6-d7e1c633380b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.671963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1014bdc-3c05-4378-b3e6-d7e1c633380b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:24.672714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1014bdc-3c05-4378-b3e6-d7e1c633380b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:24.688989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1098/2214] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:24.694494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1014bdc-3c05-4378-b3e6-d7e1c633380b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.708473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1014bdc-3c05-4378-b3e6-d7e1c633380b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.722159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211da25-78a7-4c68-97f4-cf99ad1c10a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:24.723471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211da25-78a7-4c68-97f4-cf99ad1c10a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.724467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211da25-78a7-4c68-97f4-cf99ad1c10a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.724962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211da25-78a7-4c68-97f4-cf99ad1c10a4 admin admin] PATH_INFO: `/v3/projects/ea78d9ee249e419a8041cfa69e06fb82/users/ce5708a27a274b71ad424a1cbe56c69c/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.728139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211da25-78a7-4c68-97f4-cf99ad1c10a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea78d9ee249e419a8041cfa69e06fb82, user_id=ce5708a27a274b71ad424a1cbe56c69c, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:24.732041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211da25-78a7-4c68-97f4-cf99ad1c10a4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:24.744139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1099/2215] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:24 2022] PUT /identity/v3/projects/ea78d9ee249e419a8041cfa69e06fb82/users/ce5708a27a274b71ad424a1cbe56c69c/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:24.755722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b78bc-3aee-4030-92ba-7095642a685e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:24.758000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b78bc-3aee-4030-92ba-7095642a685e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:24.758000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b78bc-3aee-4030-92ba-7095642a685e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:24.790513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-730b78bc-3aee-4030-92ba-7095642a685e None None] MFA Rules not processed for user `ce5708a27a274b71ad424a1cbe56c69c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:24.791817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-730b78bc-3aee-4030-92ba-7095642a685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:24.804009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1117/2216] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:24 2022] POST /identity/v3/users => generated 287 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:56:24.808579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2659116-1dcf-43fc-adb6-7a94232cf4c8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.828768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2659116-1dcf-43fc-adb6-7a94232cf4c8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:24.911283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:24.972527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1100/2217] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:24.981618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b78bc-3aee-4030-92ba-7095642a685e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:24.984552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-730b78bc-3aee-4030-92ba-7095642a685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:25.072678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe42eb6-873f-4b6d-95ab-5cca1c90a03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:25.073540 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42eb6-873f-4b6d-95ab-5cca1c90a03a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:25.073905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42eb6-873f-4b6d-95ab-5cca1c90a03a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:25.074286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42eb6-873f-4b6d-95ab-5cca1c90a03a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:25.075226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe42eb6-873f-4b6d-95ab-5cca1c90a03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:25.079077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe42eb6-873f-4b6d-95ab-5cca1c90a03a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:25.094526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b78bc-3aee-4030-92ba-7095642a685e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:25.100799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305bdb0-27af-4a56-aaf3-b825669c9cfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:25.101826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305bdb0-27af-4a56-aaf3-b825669c9cfc service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:25.102120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305bdb0-27af-4a56-aaf3-b825669c9cfc service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:25.102485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305bdb0-27af-4a56-aaf3-b825669c9cfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:25.108841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1118/2218] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:25.109811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe42eb6-873f-4b6d-95ab-5cca1c90a03a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:25.113766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305bdb0-27af-4a56-aaf3-b825669c9cfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:25.115503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305bdb0-27af-4a56-aaf3-b825669c9cfc service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:25.128812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1101/2219] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:56:25.136636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe42eb6-873f-4b6d-95ab-5cca1c90a03a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:25.152163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:25.152912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:25.153369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:25.153663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:25.154406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:25.156261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:25.180010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1119/2220] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:25.185280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305bdb0-27af-4a56-aaf3-b825669c9cfc service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:25.206670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:25.206670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:25.220307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305bdb0-27af-4a56-aaf3-b825669c9cfc service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:25.234112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccb5ca-b126-4211-827f-99339e92c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:25.235805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae83c9-e7dd-4841-9628-b95fbe19a42b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:25.237074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb5ca-b126-4211-827f-99339e92c82a admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:25.239029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb5ca-b126-4211-827f-99339e92c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:25.239726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccb5ca-b126-4211-827f-99339e92c82a admin admin] PATH_INFO: `/v3/projects/dd90f849d8a14b35b96b291fd2e7d472/users/1bc9e7c7f9bd4a1991beb190cf13a592/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:25.253771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:25.254836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:25.255383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:25.255910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:25.260944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:25.273235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccb5ca-b126-4211-827f-99339e92c82a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd90f849d8a14b35b96b291fd2e7d472, user_id=1bc9e7c7f9bd4a1991beb190cf13a592, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:25.279723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccb5ca-b126-4211-827f-99339e92c82a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:25.362375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:25.369714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:25.399020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1120/2221] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:25.405632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1102/2222] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:25 2022] PUT /identity/v3/projects/dd90f849d8a14b35b96b291fd2e7d472/users/1bc9e7c7f9bd4a1991beb190cf13a592/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:25.431070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b17b-b1a8-458d-858c-850580714776 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:25.439972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b17b-b1a8-458d-858c-850580714776 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:25.439972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b17b-b1a8-458d-858c-850580714776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:25.505910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b0f1b17b-b1a8-458d-858c-850580714776 None None] MFA Rules not processed for user `1bc9e7c7f9bd4a1991beb190cf13a592`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:25.507236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b0f1b17b-b1a8-458d-858c-850580714776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:25.679957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1103/2223] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:25.694935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:25.712846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c34bc-95a0-40b4-a970-0d4624ecb953 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:25.800654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515890c2-e60e-42fd-8552-a6438efc778c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:25.806057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515890c2-e60e-42fd-8552-a6438efc778c service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:25.807096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515890c2-e60e-42fd-8552-a6438efc778c service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:25.807669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515890c2-e60e-42fd-8552-a6438efc778c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:25.827862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515890c2-e60e-42fd-8552-a6438efc778c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:25.831418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515890c2-e60e-42fd-8552-a6438efc778c service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:25.864526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1121/2224] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:25.922677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1b17b-b1a8-458d-858c-850580714776 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:26.011005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1b17b-b1a8-458d-858c-850580714776 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:26.016092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:26.017454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:26.017925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:26.018381 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] PATH_INFO: `/v3/users/1bc9e7c7f9bd4a1991beb190cf13a592` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:26.019817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc9e7c7f9bd4a1991beb190cf13a592)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:26.022128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:26.034558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:26.315154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc9e7c7f9bd4a1991beb190cf13a592'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:26.372847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc9e7c7f9bd4a1991beb190cf13a592'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:26.373645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc9e7c7f9bd4a1991beb190cf13a592'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:26.374296 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc9e7c7f9bd4a1991beb190cf13a592'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:26.392012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc9e7c7f9bd4a1991beb190cf13a592'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:26.503785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1104/2225] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:25 2022] DELETE /identity/v3/users/1bc9e7c7f9bd4a1991beb190cf13a592 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:26.508089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd1caf-4650-48bb-aead-2ca616db1c08 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:26.508990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd1caf-4650-48bb-aead-2ca616db1c08 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:26.509488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd1caf-4650-48bb-aead-2ca616db1c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:26.534078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-debd1caf-4650-48bb-aead-2ca616db1c08 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:26.535735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-debd1caf-4650-48bb-aead-2ca616db1c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:26.608188 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1122/2226] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:26.624624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:26.626924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:26.737941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af305e6-31d5-4544-ad8b-22043b058961 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:26.742149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:26.742874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:26.743279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:26.743704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:26.761074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:26.762648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:26.782465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1105/2227] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:56:27.265605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:27.777701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd1caf-4650-48bb-aead-2ca616db1c08 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:27.780453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-debd1caf-4650-48bb-aead-2ca616db1c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:27.803004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd1caf-4650-48bb-aead-2ca616db1c08 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:27.836061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:27.836994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:27.837532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:27.837868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] PATH_INFO: `/v3/projects/dd90f849d8a14b35b96b291fd2e7d472` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:27.839116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd90f849d8a14b35b96b291fd2e7d472)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:27.841150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:28.015871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd90f849d8a14b35b96b291fd2e7d472', 'request_id': 'req-83eee970-6852-4dad-917c-947de2f91e97'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:28.024926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd90f849d8a14b35b96b291fd2e7d472', 'request_id': 'req-83eee970-6852-4dad-917c-947de2f91e97'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:28.130344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd90f849d8a14b35b96b291fd2e7d472', 'request_id': 'req-83eee970-6852-4dad-917c-947de2f91e97'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:28.144157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] The token cache is being invalidate because project dd90f849d8a14b35b96b291fd2e7d472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:28.144489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:28.145566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83eee970-6852-4dad-917c-947de2f91e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:28.167319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1123/2228] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:27 2022] DELETE /identity/v3/projects/dd90f849d8a14b35b96b291fd2e7d472 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:28.545852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:29.054958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:29.682916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:30.614952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:30.783242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:32.111649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:32.373521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:32.375063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:32.394295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0df66-af31-4dd4-8a36-77bcb336ac0f service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:32.467622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:32.468530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:32.468820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:32.469106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] PATH_INFO: `/v3/users/6ec33370d5634dc8a54eeeb2cf694211` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:32.470400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec33370d5634dc8a54eeeb2cf694211)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:32.472430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:32.489533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:32.801139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec33370d5634dc8a54eeeb2cf694211'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:32.817204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec33370d5634dc8a54eeeb2cf694211'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:32.817204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec33370d5634dc8a54eeeb2cf694211'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:32.817204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec33370d5634dc8a54eeeb2cf694211'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:32.825400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec33370d5634dc8a54eeeb2cf694211'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:32.918732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1106/2229] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:32 2022] DELETE /identity/v3/users/6ec33370d5634dc8a54eeeb2cf694211 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:32.920586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7af3b-30e6-4bf9-af4f-427285e9829e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:32.921191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7af3b-30e6-4bf9-af4f-427285e9829e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:32.921573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7af3b-30e6-4bf9-af4f-427285e9829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:32.968721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fce7af3b-30e6-4bf9-af4f-427285e9829e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:32.972402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fce7af3b-30e6-4bf9-af4f-427285e9829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:33.096030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1124/2230] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:33.127219 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:33.129671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:33.258758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f4c45-3e08-4750-8160-99513596137f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:33.282075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb4134-94de-45a3-8988-ac0bb792f855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:33.282909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb4134-94de-45a3-8988-ac0bb792f855 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:33.283487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb4134-94de-45a3-8988-ac0bb792f855 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:33.283874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb4134-94de-45a3-8988-ac0bb792f855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:33.299230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb4134-94de-45a3-8988-ac0bb792f855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:33.300955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb4134-94de-45a3-8988-ac0bb792f855 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:33.320576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1107/2231] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:33.838404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7af3b-30e6-4bf9-af4f-427285e9829e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:33.839871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fce7af3b-30e6-4bf9-af4f-427285e9829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:33.859780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7af3b-30e6-4bf9-af4f-427285e9829e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:33.871720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:33.872690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:33.872988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:33.873291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] PATH_INFO: `/v3/projects/941ba64c0fc344e58f5b0311597e8c77` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:33.874562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=941ba64c0fc344e58f5b0311597e8c77)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:33.876416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:34.018753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '941ba64c0fc344e58f5b0311597e8c77', 'request_id': 'req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.025158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '941ba64c0fc344e58f5b0311597e8c77', 'request_id': 'req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.082318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '941ba64c0fc344e58f5b0311597e8c77', 'request_id': 'req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.084862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:34.095759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] The token cache is being invalidate because project 941ba64c0fc344e58f5b0311597e8c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:34.096056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.096706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.098253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1125/2232] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:33 2022] DELETE /identity/v3/projects/941ba64c0fc344e58f5b0311597e8c77 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:34.102897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb4134-94de-45a3-8988-ac0bb792f855 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:34.112893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb4134-94de-45a3-8988-ac0bb792f855 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:34.189961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:34.190843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:34.191127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:34.191438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] PATH_INFO: `/v3/users/ce5708a27a274b71ad424a1cbe56c69c` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:34.192626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5708a27a274b71ad424a1cbe56c69c)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:34.194594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:34.435330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:34.452486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5708a27a274b71ad424a1cbe56c69c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.472697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5708a27a274b71ad424a1cbe56c69c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.472697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5708a27a274b71ad424a1cbe56c69c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.472697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5708a27a274b71ad424a1cbe56c69c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.477090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-97d4155f-e268-46cb-9678-7e27b188b0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5708a27a274b71ad424a1cbe56c69c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:34.544889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1108/2233] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:34 2022] DELETE /identity/v3/users/ce5708a27a274b71ad424a1cbe56c69c => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:35.009258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:35.010884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:35.027054 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372bbab-8f0a-46c7-9849-5ebec5daa5b7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:35.088250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:35.089054 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.089344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.089623 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] PATH_INFO: `/v3/projects/ea78d9ee249e419a8041cfa69e06fb82` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.090900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea78d9ee249e419a8041cfa69e06fb82)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:35.092930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:35.300102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea78d9ee249e419a8041cfa69e06fb82', 'request_id': 'req-8dbd6c93-1829-4e6b-b819-b39cb71d9835'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:35.306308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea78d9ee249e419a8041cfa69e06fb82', 'request_id': 'req-8dbd6c93-1829-4e6b-b819-b39cb71d9835'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:35.393777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea78d9ee249e419a8041cfa69e06fb82', 'request_id': 'req-8dbd6c93-1829-4e6b-b819-b39cb71d9835'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:35.405999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] The token cache is being invalidate because project ea78d9ee249e419a8041cfa69e06fb82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:35.407108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:35.407108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8dbd6c93-1829-4e6b-b819-b39cb71d9835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:35.408635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1126/2234] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:35 2022] DELETE /identity/v3/projects/ea78d9ee249e419a8041cfa69e06fb82 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:35.415041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5b636-8120-43c6-8f60-b3411e4a6f66 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.415522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5b636-8120-43c6-8f60-b3411e4a6f66 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.415637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5b636-8120-43c6-8f60-b3411e4a6f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.434270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-58c5b636-8120-43c6-8f60-b3411e4a6f66 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:35.435165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-58c5b636-8120-43c6-8f60-b3411e4a6f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:35.499074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1109/2235] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:35.506882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd55a-2cc1-498d-9054-49d063148104 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.507182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd55a-2cc1-498d-9054-49d063148104 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.507503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabd55a-2cc1-498d-9054-49d063148104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.549949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1dabd55a-2cc1-498d-9054-49d063148104 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:35.552957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1dabd55a-2cc1-498d-9054-49d063148104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:35.565049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1127/2236] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:35.569176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5b636-8120-43c6-8f60-b3411e4a6f66 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:35.590668 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5b636-8120-43c6-8f60-b3411e4a6f66 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:35.595070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6465-2bc1-4886-9c00-8014faabb5bc None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.595337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6465-2bc1-4886-9c00-8014faabb5bc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.595624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6465-2bc1-4886-9c00-8014faabb5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.596729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:35.597469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.597832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.598559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.599138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:56:35.599565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:35.602702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:35.623071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-935c6465-2bc1-4886-9c00-8014faabb5bc None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:35.624795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-935c6465-2bc1-4886-9c00-8014faabb5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:35.626454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1110/2237] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:56:35.632243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:35.646528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341d79e-f949-45a4-9d2a-f19a1ae125fb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:35.657829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2952759-5059-404e-af5e-85c37303508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:35.658529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2952759-5059-404e-af5e-85c37303508d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.658824 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2952759-5059-404e-af5e-85c37303508d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.659106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2952759-5059-404e-af5e-85c37303508d admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.659858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2952759-5059-404e-af5e-85c37303508d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:35.661813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2952759-5059-404e-af5e-85c37303508d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:35.698526 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1128/2238] 10.222.0.98 () {58 vars in 967 bytes} [Fri Jul 15 15:56:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:35.710931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c6465-2bc1-4886-9c00-8014faabb5bc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:35.713164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-935c6465-2bc1-4886-9c00-8014faabb5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:35.740258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:35.764149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1111/2239] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:35 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:35.768202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2952759-5059-404e-af5e-85c37303508d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:35.781982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c6465-2bc1-4886-9c00-8014faabb5bc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:35.829863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2952759-5059-404e-af5e-85c37303508d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:35.835174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb88d6a-4ef8-4b80-a49c-b327b198bdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:35.836796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb88d6a-4ef8-4b80-a49c-b327b198bdbc admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.837395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb88d6a-4ef8-4b80-a49c-b327b198bdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.837682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb88d6a-4ef8-4b80-a49c-b327b198bdbc admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.838390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb88d6a-4ef8-4b80-a49c-b327b198bdbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:35.840464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb88d6a-4ef8-4b80-a49c-b327b198bdbc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:35.842247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96867ca-f90d-4c49-9de3-8b6490549ef0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:35.842569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96867ca-f90d-4c49-9de3-8b6490549ef0 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.842856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96867ca-f90d-4c49-9de3-8b6490549ef0 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.843200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96867ca-f90d-4c49-9de3-8b6490549ef0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.900010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96867ca-f90d-4c49-9de3-8b6490549ef0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:35.901611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96867ca-f90d-4c49-9de3-8b6490549ef0 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:35.909654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1129/2240] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:56:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 15:56:35.925221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1112/2241] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:35 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:56:35.929115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96867ca-f90d-4c49-9de3-8b6490549ef0 service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:35.939805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96867ca-f90d-4c49-9de3-8b6490549ef0 service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:35.944635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe73b68-59d5-48ff-826c-c0cb88bbb93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:35.946815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe73b68-59d5-48ff-826c-c0cb88bbb93f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.946815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe73b68-59d5-48ff-826c-c0cb88bbb93f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.946815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe73b68-59d5-48ff-826c-c0cb88bbb93f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.947233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe73b68-59d5-48ff-826c-c0cb88bbb93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:35.949207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe73b68-59d5-48ff-826c-c0cb88bbb93f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:35.959880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1130/2242] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:35.964847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb88d6a-4ef8-4b80-a49c-b327b198bdbc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:35.975589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb88d6a-4ef8-4b80-a49c-b327b198bdbc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:35.980567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a2d19-c0aa-414f-9283-6714def7bf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:35.981241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a2d19-c0aa-414f-9283-6714def7bf55 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:35.981531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a2d19-c0aa-414f-9283-6714def7bf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:35.981815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a2d19-c0aa-414f-9283-6714def7bf55 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:35.982550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a2d19-c0aa-414f-9283-6714def7bf55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:35.985871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a2d19-c0aa-414f-9283-6714def7bf55 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:35.993596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:35.999707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1113/2243] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:36.003962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe73b68-59d5-48ff-826c-c0cb88bbb93f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:36.015605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe73b68-59d5-48ff-826c-c0cb88bbb93f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:36.021009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9942-d9cb-4384-9368-a132d9f3ce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:36.021887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9942-d9cb-4384-9368-a132d9f3ce9b admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:36.022171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9942-d9cb-4384-9368-a132d9f3ce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:36.022448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c9942-d9cb-4384-9368-a132d9f3ce9b admin admin] PATH_INFO: `/v3/projects/9667e9f3071045f7ae39a307477a7dcb/users/e6291d9d84da4c938e3a82f3cc5b8454/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:36.039045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c9942-d9cb-4384-9368-a132d9f3ce9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9667e9f3071045f7ae39a307477a7dcb, user_id=e6291d9d84da4c938e3a82f3cc5b8454, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:36.041800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c9942-d9cb-4384-9368-a132d9f3ce9b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:36.154305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1131/2244] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:36 2022] PUT /identity/v3/projects/9667e9f3071045f7ae39a307477a7dcb/users/e6291d9d84da4c938e3a82f3cc5b8454/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:36.165406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e3ca-ab4f-40c8-a51e-1641ad491ded None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:36.165702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e3ca-ab4f-40c8-a51e-1641ad491ded None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:36.165986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e3ca-ab4f-40c8-a51e-1641ad491ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:36.196697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-cc76e3ca-ab4f-40c8-a51e-1641ad491ded None None] MFA Rules not processed for user `e6291d9d84da4c938e3a82f3cc5b8454`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:36.196697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cc76e3ca-ab4f-40c8-a51e-1641ad491ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:36.268622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1114/2245] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:36.272639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9942-d9cb-4384-9368-a132d9f3ce9b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:36.316826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:36.346111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c9942-d9cb-4384-9368-a132d9f3ce9b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:36.352503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7ca57-139b-4c16-b220-afe395941823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:36.354639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7ca57-139b-4c16-b220-afe395941823 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:36.354639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7ca57-139b-4c16-b220-afe395941823 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:36.354639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7ca57-139b-4c16-b220-afe395941823 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:36.354639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7ca57-139b-4c16-b220-afe395941823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:36.356482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7ca57-139b-4c16-b220-afe395941823 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:36.446570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1132/2246] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:36 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:56:36.450611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e3ca-ab4f-40c8-a51e-1641ad491ded None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:36.462250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e3ca-ab4f-40c8-a51e-1641ad491ded None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:36.521269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f45a59-f316-4197-94b0-ff2736cb75af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:36.522071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45a59-f316-4197-94b0-ff2736cb75af admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:36.522370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45a59-f316-4197-94b0-ff2736cb75af admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:36.522681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45a59-f316-4197-94b0-ff2736cb75af admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:36.523406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f45a59-f316-4197-94b0-ff2736cb75af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:36.526001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f45a59-f316-4197-94b0-ff2736cb75af admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:36.618910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1115/2247] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:36 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:36.622858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7ca57-139b-4c16-b220-afe395941823 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:36.634459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7ca57-139b-4c16-b220-afe395941823 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:36.639120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde276b-dd8b-4098-9d8b-e8aed9ae6d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:36.639980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde276b-dd8b-4098-9d8b-e8aed9ae6d2e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:36.640197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde276b-dd8b-4098-9d8b-e8aed9ae6d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:36.640476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde276b-dd8b-4098-9d8b-e8aed9ae6d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:36.641203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde276b-dd8b-4098-9d8b-e8aed9ae6d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:36.642800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde276b-dd8b-4098-9d8b-e8aed9ae6d2e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:36.654895 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1133/2248] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:36.660158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f45a59-f316-4197-94b0-ff2736cb75af admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:36.669728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f45a59-f316-4197-94b0-ff2736cb75af admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:36.674461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38245471-591b-440c-8518-2070ddfe3336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:36.675384 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38245471-591b-440c-8518-2070ddfe3336 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:36.675677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38245471-591b-440c-8518-2070ddfe3336 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:36.675958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38245471-591b-440c-8518-2070ddfe3336 admin admin] PATH_INFO: `/v3/projects/f7b3cff299b34c87b2aa52ea9f71d58f/users/f89543d59e274767ab7de7ae76f05501/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:36.692604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38245471-591b-440c-8518-2070ddfe3336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b3cff299b34c87b2aa52ea9f71d58f, user_id=f89543d59e274767ab7de7ae76f05501, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:36.696053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38245471-591b-440c-8518-2070ddfe3336 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:36.860019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1116/2249] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:36 2022] PUT /identity/v3/projects/f7b3cff299b34c87b2aa52ea9f71d58f/users/f89543d59e274767ab7de7ae76f05501/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:36.864374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde276b-dd8b-4098-9d8b-e8aed9ae6d2e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:36.876869 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde276b-dd8b-4098-9d8b-e8aed9ae6d2e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:36.940226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86873c35-271e-43f8-803a-8849783f7599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:36.940906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873c35-271e-43f8-803a-8849783f7599 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:36.942547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873c35-271e-43f8-803a-8849783f7599 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:36.942547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873c35-271e-43f8-803a-8849783f7599 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:36.942547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86873c35-271e-43f8-803a-8849783f7599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:36.944116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86873c35-271e-43f8-803a-8849783f7599 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:36.954707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1134/2250] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:36.958743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38245471-591b-440c-8518-2070ddfe3336 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:36.970477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38245471-591b-440c-8518-2070ddfe3336 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:36.975257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:36.976177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:36.978505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:36.978505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] PATH_INFO: `/v3/projects/f7b3cff299b34c87b2aa52ea9f71d58f/users/f89543d59e274767ab7de7ae76f05501/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:36.979525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b3cff299b34c87b2aa52ea9f71d58f, user_id=f89543d59e274767ab7de7ae76f05501, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:36.982236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:36.993083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1117/2251] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:36 2022] PUT /identity/v3/projects/f7b3cff299b34c87b2aa52ea9f71d58f/users/f89543d59e274767ab7de7ae76f05501/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:37.002977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eab33e-4de7-4e7e-ae0e-62522b5b82fb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:37.003402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eab33e-4de7-4e7e-ae0e-62522b5b82fb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:37.003759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eab33e-4de7-4e7e-ae0e-62522b5b82fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:37.033222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-74eab33e-4de7-4e7e-ae0e-62522b5b82fb None None] MFA Rules not processed for user `f89543d59e274767ab7de7ae76f05501`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:37.040010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-74eab33e-4de7-4e7e-ae0e-62522b5b82fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:37.108275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1135/2252] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:37.119672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:37.121491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:37.167511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:37.184646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8f13e-756b-4f5a-b1db-9d7355f07e1d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:37.188581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff7f0f-3593-47e9-8922-cfa62b19d4a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:37.189201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7f0f-3593-47e9-8922-cfa62b19d4a4 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:37.189489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7f0f-3593-47e9-8922-cfa62b19d4a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:37.189770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7f0f-3593-47e9-8922-cfa62b19d4a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:37.199396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff7f0f-3593-47e9-8922-cfa62b19d4a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:37.200909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff7f0f-3593-47e9-8922-cfa62b19d4a4 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:37.210598 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1118/2253] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:38.563848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eab33e-4de7-4e7e-ae0e-62522b5b82fb None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:38.596324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:38.627132 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eab33e-4de7-4e7e-ae0e-62522b5b82fb None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:38.632240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:38.633108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:38.633399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:38.633683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] PATH_INFO: `/v3/users/e6291d9d84da4c938e3a82f3cc5b8454` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:38.634993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6291d9d84da4c938e3a82f3cc5b8454)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:38.637032 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:38.866532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6291d9d84da4c938e3a82f3cc5b8454'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:38.879181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6291d9d84da4c938e3a82f3cc5b8454'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:38.879921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6291d9d84da4c938e3a82f3cc5b8454'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:38.880573 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6291d9d84da4c938e3a82f3cc5b8454'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:38.890070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6291d9d84da4c938e3a82f3cc5b8454'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:38.993418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1136/2254] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:38 2022] DELETE /identity/v3/users/e6291d9d84da4c938e3a82f3cc5b8454 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:38.998040 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe2d71-6a1d-4793-a7c7-a74825d00ea7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:38.998507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe2d71-6a1d-4793-a7c7-a74825d00ea7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:38.998636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe2d71-6a1d-4793-a7c7-a74825d00ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:39.019201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f7fe2d71-6a1d-4793-a7c7-a74825d00ea7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:39.019201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f7fe2d71-6a1d-4793-a7c7-a74825d00ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:39.095271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1119/2255] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:39.104323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:39.107746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:39.180304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd696b-ccdf-4f7d-b46d-f62cb4fc1cf7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:39.184456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:39.185327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:39.185818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:39.186263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:39.197360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:39.198825 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:39.210525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1137/2256] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:39.518922 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe2d71-6a1d-4793-a7c7-a74825d00ea7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:39.520430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f7fe2d71-6a1d-4793-a7c7-a74825d00ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:39.532683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe2d71-6a1d-4793-a7c7-a74825d00ea7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:39.537631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:39.539628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:39.539628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:39.539628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] PATH_INFO: `/v3/users/a3efd91d94b44c6aa41087c2f017861f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:39.540827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3efd91d94b44c6aa41087c2f017861f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:39.542924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:39.809127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:39.810581 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:39.821261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3efd91d94b44c6aa41087c2f017861f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:39.833622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e92c20-ad68-44cf-a06b-366c4e7e97e3 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:39.839484 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3efd91d94b44c6aa41087c2f017861f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:39.840339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3efd91d94b44c6aa41087c2f017861f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:39.841061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3efd91d94b44c6aa41087c2f017861f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:39.853480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3efd91d94b44c6aa41087c2f017861f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:39.865985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:39.866829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:39.867873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:39.886745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] PATH_INFO: `/v3/projects/9667e9f3071045f7ae39a307477a7dcb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:39.886745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9667e9f3071045f7ae39a307477a7dcb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:39.886745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:39.998054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1120/2257] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:39 2022] DELETE /identity/v3/users/a3efd91d94b44c6aa41087c2f017861f => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:40.019193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:40.024182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9667e9f3071045f7ae39a307477a7dcb', 'request_id': 'req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.030472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9667e9f3071045f7ae39a307477a7dcb', 'request_id': 'req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.125520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9667e9f3071045f7ae39a307477a7dcb', 'request_id': 'req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.153607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] The token cache is being invalidate because project 9667e9f3071045f7ae39a307477a7dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:40.154160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.155000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.156793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1138/2258] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:39 2022] DELETE /identity/v3/projects/9667e9f3071045f7ae39a307477a7dcb => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:40.162717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:40.165039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:40.241957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:40.276189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fac61-f8fd-4fa4-94df-b02342e0f377 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:40.389846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:40.390911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:40.391209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:40.391583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] PATH_INFO: `/v3/users/f89543d59e274767ab7de7ae76f05501` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:40.392844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f89543d59e274767ab7de7ae76f05501)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:40.395567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:40.470088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:40.471699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:40.486152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cbd79-36d5-40d8-96e9-c3adc8998b88 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:40.504537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:40.505170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:40.505462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:40.505744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] PATH_INFO: `/v3/projects/8d7726f6c7444f16ac5ea9696eacdf57` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:40.507603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7726f6c7444f16ac5ea9696eacdf57)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:40.511453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:40.835152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f89543d59e274767ab7de7ae76f05501'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.835352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7726f6c7444f16ac5ea9696eacdf57', 'request_id': 'req-5a345071-cee7-4d58-90ec-dc1cc5a4318c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.835352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7726f6c7444f16ac5ea9696eacdf57', 'request_id': 'req-5a345071-cee7-4d58-90ec-dc1cc5a4318c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.836329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f89543d59e274767ab7de7ae76f05501'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.836329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f89543d59e274767ab7de7ae76f05501'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.836329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f89543d59e274767ab7de7ae76f05501'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.836329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f89543d59e274767ab7de7ae76f05501'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.916127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7726f6c7444f16ac5ea9696eacdf57', 'request_id': 'req-5a345071-cee7-4d58-90ec-dc1cc5a4318c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.928019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] The token cache is being invalidate because project 8d7726f6c7444f16ac5ea9696eacdf57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:40.928310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.928939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5a345071-cee7-4d58-90ec-dc1cc5a4318c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:40.930545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1139/2259] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:40 2022] DELETE /identity/v3/projects/8d7726f6c7444f16ac5ea9696eacdf57 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:40.935434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1091c-4934-49a4-beda-aae3d488a7e0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:40.935710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1091c-4934-49a4-beda-aae3d488a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:40.935990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1091c-4934-49a4-beda-aae3d488a7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:40.952381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c8e1091c-4934-49a4-beda-aae3d488a7e0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:40.953582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c8e1091c-4934-49a4-beda-aae3d488a7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:41.010635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1140/2260] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:41.018418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424ad52-9475-4da3-b027-5d5d4d402c55 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.018710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424ad52-9475-4da3-b027-5d5d4d402c55 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.018993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424ad52-9475-4da3-b027-5d5d4d402c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.036981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9424ad52-9475-4da3-b027-5d5d4d402c55 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:41.041407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9424ad52-9475-4da3-b027-5d5d4d402c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:41.046570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1141/2261] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:41.050578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424ad52-9475-4da3-b027-5d5d4d402c55 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:41.059323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424ad52-9475-4da3-b027-5d5d4d402c55 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:41.063931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:41.064553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.064840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.065122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.065836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:56:41.066147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:41.067779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:41.074071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1121/2262] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:40 2022] DELETE /identity/v3/users/f89543d59e274767ab7de7ae76f05501 => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:41.093135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1142/2263] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:56:41.096222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:41.097740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:41.176904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb16c-affb-4067-8824-37411e369fc3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:41.180862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe17ba-7f0d-46ff-8546-d6cf449268ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:41.182183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe17ba-7f0d-46ff-8546-d6cf449268ff admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.182473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe17ba-7f0d-46ff-8546-d6cf449268ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.182884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe17ba-7f0d-46ff-8546-d6cf449268ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.183565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe17ba-7f0d-46ff-8546-d6cf449268ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:41.185491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe17ba-7f0d-46ff-8546-d6cf449268ff admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:41.287311 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1122/2264] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:41 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:41.291736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:41.304052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a6a8-f22a-4f80-b484-5b943dcb1556 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:41.403673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8cf00-1c74-427f-aacc-e2526cc00ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:41.404374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8cf00-1c74-427f-aacc-e2526cc00ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.404648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8cf00-1c74-427f-aacc-e2526cc00ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.404942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8cf00-1c74-427f-aacc-e2526cc00ad7 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.405618 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8cf00-1c74-427f-aacc-e2526cc00ad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:41.407648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8cf00-1c74-427f-aacc-e2526cc00ad7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:41.498009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1143/2265] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:41 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:41.499441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe17ba-7f0d-46ff-8546-d6cf449268ff admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:41.512529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe17ba-7f0d-46ff-8546-d6cf449268ff admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:41.524243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:41.525076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.525391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.525634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.526531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:41.540258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:41.564569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8cf00-1c74-427f-aacc-e2526cc00ad7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:41.575893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1123/2266] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:41.576962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:41.580373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8cf00-1c74-427f-aacc-e2526cc00ad7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:41.584139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:41.588180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f3c4c-3b4e-4dcf-88c4-c3c6f192bbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:41.599221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f3c4c-3b4e-4dcf-88c4-c3c6f192bbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.599221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f3c4c-3b4e-4dcf-88c4-c3c6f192bbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.599221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f3c4c-3b4e-4dcf-88c4-c3c6f192bbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.599221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f3c4c-3b4e-4dcf-88c4-c3c6f192bbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:41.599221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f3c4c-3b4e-4dcf-88c4-c3c6f192bbc9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:41.603629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70362a4-43ab-4d9c-a2fc-4dbba35eecc7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:41.604373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1144/2267] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:41.608921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f3c4c-3b4e-4dcf-88c4-c3c6f192bbc9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:41.612320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:41.613600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.613874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.614257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] PATH_INFO: `/v3/projects/f7b3cff299b34c87b2aa52ea9f71d58f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.619112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b3cff299b34c87b2aa52ea9f71d58f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:41.620632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f3c4c-3b4e-4dcf-88c4-c3c6f192bbc9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:41.621385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:41.627847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741c983-be75-4f9f-bc86-2415276dde42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:41.641803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741c983-be75-4f9f-bc86-2415276dde42 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.642718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741c983-be75-4f9f-bc86-2415276dde42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.643474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741c983-be75-4f9f-bc86-2415276dde42 admin admin] PATH_INFO: `/v3/projects/cd50bd3524414dff85babb547c38f2bb/users/6540a053913a41f0868923ae10cb6c46/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.663356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741c983-be75-4f9f-bc86-2415276dde42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd50bd3524414dff85babb547c38f2bb, user_id=6540a053913a41f0868923ae10cb6c46, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:41.666267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741c983-be75-4f9f-bc86-2415276dde42 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:41.830392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1145/2268] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:41 2022] PUT /identity/v3/projects/cd50bd3524414dff85babb547c38f2bb/users/6540a053913a41f0868923ae10cb6c46/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:41.842364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0c896-6064-426f-b9cc-7c0b7435cd9b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:41.843026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0c896-6064-426f-b9cc-7c0b7435cd9b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:41.843128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b3cff299b34c87b2aa52ea9f71d58f', 'request_id': 'req-6bb1da71-c11e-4055-8427-91d3cfcd031e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:41.843235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0c896-6064-426f-b9cc-7c0b7435cd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:41.849345 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b3cff299b34c87b2aa52ea9f71d58f', 'request_id': 'req-6bb1da71-c11e-4055-8427-91d3cfcd031e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:41.854845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:41.857354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b3cff299b34c87b2aa52ea9f71d58f', 'request_id': 'req-6bb1da71-c11e-4055-8427-91d3cfcd031e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:41.874867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5cd0c896-6064-426f-b9cc-7c0b7435cd9b None None] MFA Rules not processed for user `6540a053913a41f0868923ae10cb6c46`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:41.876303 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5cd0c896-6064-426f-b9cc-7c0b7435cd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:41.942250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1146/2269] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:41.953638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0c896-6064-426f-b9cc-7c0b7435cd9b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:41.955102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5cd0c896-6064-426f-b9cc-7c0b7435cd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:42.001144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] The token cache is being invalidate because project f7b3cff299b34c87b2aa52ea9f71d58f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:42.001539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:42.002235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6bb1da71-c11e-4055-8427-91d3cfcd031e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:42.003896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1124/2270] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:41 2022] DELETE /identity/v3/projects/f7b3cff299b34c87b2aa52ea9f71d58f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:42.008741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7106276-a9f7-4287-94d9-de9ce77d604f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.009041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7106276-a9f7-4287-94d9-de9ce77d604f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.009321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7106276-a9f7-4287-94d9-de9ce77d604f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.012826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0c896-6064-426f-b9cc-7c0b7435cd9b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:42.017434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4aa7c-0b8c-40a6-a460-852e6a67fccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:42.018090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4aa7c-0b8c-40a6-a460-852e6a67fccc service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.018584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4aa7c-0b8c-40a6-a460-852e6a67fccc service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.018876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4aa7c-0b8c-40a6-a460-852e6a67fccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.020744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-deb4aa7c-0b8c-40a6-a460-852e6a67fccc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:42.027114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a7106276-a9f7-4287-94d9-de9ce77d604f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:42.028325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a7106276-a9f7-4287-94d9-de9ce77d604f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:42.034277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4aa7c-0b8c-40a6-a460-852e6a67fccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:42.035873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4aa7c-0b8c-40a6-a460-852e6a67fccc service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:42.046201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1147/2271] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:56:42.091681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1125/2272] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:42.099730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b1f4c-09de-4160-a2a7-a5442e81c2fc None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.100096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b1f4c-09de-4160-a2a7-a5442e81c2fc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.100450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b1f4c-09de-4160-a2a7-a5442e81c2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.120107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ac4b1f4c-09de-4160-a2a7-a5442e81c2fc None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:42.121318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ac4b1f4c-09de-4160-a2a7-a5442e81c2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:42.129227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1148/2273] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:42.145936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7106276-a9f7-4287-94d9-de9ce77d604f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:42.155954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7106276-a9f7-4287-94d9-de9ce77d604f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:42.160580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:42.161220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.161503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.161782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.162506 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:56:42.162819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:42.165042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:42.176754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1126/2274] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:56:42.181082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b1f4c-09de-4160-a2a7-a5442e81c2fc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:42.189995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b1f4c-09de-4160-a2a7-a5442e81c2fc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:42.200244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83041-266a-49ee-97d5-089b67f989b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:42.200244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83041-266a-49ee-97d5-089b67f989b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.200244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83041-266a-49ee-97d5-089b67f989b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.200244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83041-266a-49ee-97d5-089b67f989b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.200661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a83041-266a-49ee-97d5-089b67f989b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:42.200908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a83041-266a-49ee-97d5-089b67f989b0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:42.345232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:42.352560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1149/2275] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:42 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:42.356953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:42.369133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3a31c-59fa-4647-a8b2-c664325611b5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:42.440016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cddb46-c179-4956-891a-f37ac978b437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:42.441403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddb46-c179-4956-891a-f37ac978b437 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.441947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddb46-c179-4956-891a-f37ac978b437 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.442451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddb46-c179-4956-891a-f37ac978b437 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.444265 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cddb46-c179-4956-891a-f37ac978b437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:42.446621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cddb46-c179-4956-891a-f37ac978b437 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:42.542734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1127/2276] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:42 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:56:42.546911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83041-266a-49ee-97d5-089b67f989b0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:42.557403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83041-266a-49ee-97d5-089b67f989b0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:42.562346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bd402-701c-4cef-acf4-37f76ba45627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:42.563008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bd402-701c-4cef-acf4-37f76ba45627 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.563367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bd402-701c-4cef-acf4-37f76ba45627 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.563750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bd402-701c-4cef-acf4-37f76ba45627 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.564387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bd402-701c-4cef-acf4-37f76ba45627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:42.565975 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2bd402-701c-4cef-acf4-37f76ba45627 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:42.576898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1150/2277] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:42.581038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cddb46-c179-4956-891a-f37ac978b437 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:42.591915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cddb46-c179-4956-891a-f37ac978b437 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:42.596725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91047f35-ec40-4e0f-afeb-97229352f607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:42.597389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047f35-ec40-4e0f-afeb-97229352f607 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.597683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047f35-ec40-4e0f-afeb-97229352f607 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.597964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047f35-ec40-4e0f-afeb-97229352f607 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.598714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91047f35-ec40-4e0f-afeb-97229352f607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:42.600405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91047f35-ec40-4e0f-afeb-97229352f607 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:42.610569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1128/2278] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:42.619319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bd402-701c-4cef-acf4-37f76ba45627 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:42.629345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bd402-701c-4cef-acf4-37f76ba45627 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:42.634488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82fc18-b102-46af-abb4-71767e8c61e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:42.635531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82fc18-b102-46af-abb4-71767e8c61e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.635846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82fc18-b102-46af-abb4-71767e8c61e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.636136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82fc18-b102-46af-abb4-71767e8c61e9 admin admin] PATH_INFO: `/v3/projects/e50108c9ad094d14a3913bd2f936a447/users/1391861053c24dafb0c9440be892a1cf/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.651005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82fc18-b102-46af-abb4-71767e8c61e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50108c9ad094d14a3913bd2f936a447, user_id=1391861053c24dafb0c9440be892a1cf, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:42.653720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82fc18-b102-46af-abb4-71767e8c61e9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:42.762503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1151/2279] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:42 2022] PUT /identity/v3/projects/e50108c9ad094d14a3913bd2f936a447/users/1391861053c24dafb0c9440be892a1cf/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:56:42.832480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf92e-a73c-4fb1-925c-e83d15bcb68b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:42.834589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf92e-a73c-4fb1-925c-e83d15bcb68b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:42.834589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf92e-a73c-4fb1-925c-e83d15bcb68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:42.872210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-42abf92e-a73c-4fb1-925c-e83d15bcb68b None None] MFA Rules not processed for user `1391861053c24dafb0c9440be892a1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:42.873448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-42abf92e-a73c-4fb1-925c-e83d15bcb68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:42.943343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1129/2280] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:42.948349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82fc18-b102-46af-abb4-71767e8c61e9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:43.018177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82fc18-b102-46af-abb4-71767e8c61e9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:43.022994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6592-8168-4410-a212-a62c61b6cba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:43.026891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6592-8168-4410-a212-a62c61b6cba0 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:43.026891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6592-8168-4410-a212-a62c61b6cba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:43.026891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b6592-8168-4410-a212-a62c61b6cba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:43.026891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b6592-8168-4410-a212-a62c61b6cba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:43.027743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b6592-8168-4410-a212-a62c61b6cba0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:43.061340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:43.154935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1152/2281] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:42 2022] POST /identity/v3/projects => generated 383 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:56:43.159121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abf92e-a73c-4fb1-925c-e83d15bcb68b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:43.170051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abf92e-a73c-4fb1-925c-e83d15bcb68b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:43.227133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801c1d5-4828-46fc-9656-0c669ce0faf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:43.227945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801c1d5-4828-46fc-9656-0c669ce0faf9 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:43.228236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801c1d5-4828-46fc-9656-0c669ce0faf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:43.228517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801c1d5-4828-46fc-9656-0c669ce0faf9 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:43.229201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801c1d5-4828-46fc-9656-0c669ce0faf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:43.231165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801c1d5-4828-46fc-9656-0c669ce0faf9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:43.250521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:43.318547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1130/2282] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:43 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:43.322668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6592-8168-4410-a212-a62c61b6cba0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:43.333250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b6592-8168-4410-a212-a62c61b6cba0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:43.338451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07b7f-6828-4927-85f7-01dc95382ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:43.339106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07b7f-6828-4927-85f7-01dc95382ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:43.339428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07b7f-6828-4927-85f7-01dc95382ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:43.339711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07b7f-6828-4927-85f7-01dc95382ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:43.340481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e07b7f-6828-4927-85f7-01dc95382ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:43.342097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e07b7f-6828-4927-85f7-01dc95382ef6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:43.353142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1153/2283] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:56:43.357503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801c1d5-4828-46fc-9656-0c669ce0faf9 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:43.368711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801c1d5-4828-46fc-9656-0c669ce0faf9 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:43.373475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f51d52-51a4-45ac-9650-d0a7d02ca0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:43.375244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51d52-51a4-45ac-9650-d0a7d02ca0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:43.375244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51d52-51a4-45ac-9650-d0a7d02ca0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:43.375244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f51d52-51a4-45ac-9650-d0a7d02ca0e9 admin admin] PATH_INFO: `/v3/projects/15e6b1fa2ab74eaca163b2928d6c85df/users/8c1110c084e54907a2c4eae2e146f788/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:43.389935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f51d52-51a4-45ac-9650-d0a7d02ca0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e6b1fa2ab74eaca163b2928d6c85df, user_id=8c1110c084e54907a2c4eae2e146f788, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:43.392692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f51d52-51a4-45ac-9650-d0a7d02ca0e9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:43.479053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1131/2284] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:43 2022] PUT /identity/v3/projects/15e6b1fa2ab74eaca163b2928d6c85df/users/8c1110c084e54907a2c4eae2e146f788/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:43.483334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07b7f-6828-4927-85f7-01dc95382ef6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:43.494707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07b7f-6828-4927-85f7-01dc95382ef6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:43.549819 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b11ae-176a-4b7a-b899-49aa343baa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:43.550595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b11ae-176a-4b7a-b899-49aa343baa20 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:43.550766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b11ae-176a-4b7a-b899-49aa343baa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:43.551044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b11ae-176a-4b7a-b899-49aa343baa20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:43.551798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b11ae-176a-4b7a-b899-49aa343baa20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:43.553391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b11ae-176a-4b7a-b899-49aa343baa20 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:43.565664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1154/2285] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:43.569914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f51d52-51a4-45ac-9650-d0a7d02ca0e9 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:43.580411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f51d52-51a4-45ac-9650-d0a7d02ca0e9 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:43.585250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:43.586138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:43.586425 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:43.586710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] PATH_INFO: `/v3/projects/15e6b1fa2ab74eaca163b2928d6c85df/users/8c1110c084e54907a2c4eae2e146f788/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:43.588991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e6b1fa2ab74eaca163b2928d6c85df, user_id=8c1110c084e54907a2c4eae2e146f788, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:43.591743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:43.603197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1132/2286] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:43 2022] PUT /identity/v3/projects/15e6b1fa2ab74eaca163b2928d6c85df/users/8c1110c084e54907a2c4eae2e146f788/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:43.613547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16548457-abae-4923-afb6-12acfd585ca6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:43.613841 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16548457-abae-4923-afb6-12acfd585ca6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:43.614118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16548457-abae-4923-afb6-12acfd585ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:43.647042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-16548457-abae-4923-afb6-12acfd585ca6 None None] MFA Rules not processed for user `8c1110c084e54907a2c4eae2e146f788`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:43.649604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-16548457-abae-4923-afb6-12acfd585ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:43.670952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:43.723320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1155/2287] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:43.736050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:43.738129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:43.771248 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:43.805268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7124fd92-cf71-4092-b543-8afd861d0fde admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:43.810165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b21f6-b9ec-470a-99ca-64d11e6bf534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:43.810892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b21f6-b9ec-470a-99ca-64d11e6bf534 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:43.811571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b21f6-b9ec-470a-99ca-64d11e6bf534 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:43.812084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b21f6-b9ec-470a-99ca-64d11e6bf534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:43.822566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b21f6-b9ec-470a-99ca-64d11e6bf534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:43.824041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b21f6-b9ec-470a-99ca-64d11e6bf534 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:43.837250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1133/2288] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:44.092904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16548457-abae-4923-afb6-12acfd585ca6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:44.103277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16548457-abae-4923-afb6-12acfd585ca6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:44.107160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f3b5f-5ba5-4dc0-81b0-3fe1e57bf577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:44.107920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f3b5f-5ba5-4dc0-81b0-3fe1e57bf577 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:44.108277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f3b5f-5ba5-4dc0-81b0-3fe1e57bf577 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:44.108628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f3b5f-5ba5-4dc0-81b0-3fe1e57bf577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:44.172018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f3b5f-5ba5-4dc0-81b0-3fe1e57bf577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:44.173468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f3b5f-5ba5-4dc0-81b0-3fe1e57bf577 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:44.182650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1156/2289] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:44.747750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b21f6-b9ec-470a-99ca-64d11e6bf534 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:44.820496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b21f6-b9ec-470a-99ca-64d11e6bf534 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:44.825492 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:44.826423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:44.826698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:44.826977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] PATH_INFO: `/v3/users/1391861053c24dafb0c9440be892a1cf` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:44.828227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=1391861053c24dafb0c9440be892a1cf)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:44.830178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:45.049238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1391861053c24dafb0c9440be892a1cf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:45.073303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1391861053c24dafb0c9440be892a1cf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:45.073303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1391861053c24dafb0c9440be892a1cf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:45.073303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1391861053c24dafb0c9440be892a1cf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:45.078198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1391861053c24dafb0c9440be892a1cf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:45.229185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1134/2290] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:44 2022] DELETE /identity/v3/users/1391861053c24dafb0c9440be892a1cf => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:45.234178 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e859fa-0e4b-4014-97e7-7d43894c7b04 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:45.235706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e859fa-0e4b-4014-97e7-7d43894c7b04 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:45.235706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e859fa-0e4b-4014-97e7-7d43894c7b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:45.260061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-82e859fa-0e4b-4014-97e7-7d43894c7b04 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:45.261349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-82e859fa-0e4b-4014-97e7-7d43894c7b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:45.323643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1157/2291] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:45.338218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:45.339754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:45.416496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:45.421857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a19f47-9b20-4f91-b05d-317dc7f60ada admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:45.425634 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a58fd2-0caa-453e-860c-1fd910c96369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:45.426283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a58fd2-0caa-453e-860c-1fd910c96369 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:45.426564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a58fd2-0caa-453e-860c-1fd910c96369 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:45.426868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a58fd2-0caa-453e-860c-1fd910c96369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:45.436972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a58fd2-0caa-453e-860c-1fd910c96369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:45.438416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a58fd2-0caa-453e-860c-1fd910c96369 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:45.453707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1135/2292] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:45.861829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e859fa-0e4b-4014-97e7-7d43894c7b04 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:45.863343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-82e859fa-0e4b-4014-97e7-7d43894c7b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:45.879862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e859fa-0e4b-4014-97e7-7d43894c7b04 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:45.885936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:45.886822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:45.887110 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:45.887425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] PATH_INFO: `/v3/projects/e50108c9ad094d14a3913bd2f936a447` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:45.889698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50108c9ad094d14a3913bd2f936a447)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:45.891925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:46.034673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50108c9ad094d14a3913bd2f936a447', 'request_id': 'req-94a4de08-465e-4504-9179-4402ae4ae945'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.036108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:46.042760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50108c9ad094d14a3913bd2f936a447', 'request_id': 'req-94a4de08-465e-4504-9179-4402ae4ae945'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.144462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50108c9ad094d14a3913bd2f936a447', 'request_id': 'req-94a4de08-465e-4504-9179-4402ae4ae945'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.172844 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] The token cache is being invalidate because project e50108c9ad094d14a3913bd2f936a447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:46.172959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.173707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.176395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1158/2293] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:45 2022] DELETE /identity/v3/projects/e50108c9ad094d14a3913bd2f936a447 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:46.180473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a58fd2-0caa-453e-860c-1fd910c96369 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:46.191589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a58fd2-0caa-453e-860c-1fd910c96369 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:46.255109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:46.256092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:46.256378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:46.256947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] PATH_INFO: `/v3/users/8c1110c084e54907a2c4eae2e146f788` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:46.258705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1110c084e54907a2c4eae2e146f788)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:46.262978 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:46.436997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:46.438554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:46.455087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4de08-465e-4504-9179-4402ae4ae945 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:46.460083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:46.462273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:46.462751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:46.463179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] PATH_INFO: `/v3/users/6540a053913a41f0868923ae10cb6c46` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:46.464632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6540a053913a41f0868923ae10cb6c46)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:46.466774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:46.537097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1110c084e54907a2c4eae2e146f788'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.555067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1110c084e54907a2c4eae2e146f788'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.556095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1110c084e54907a2c4eae2e146f788'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.556766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1110c084e54907a2c4eae2e146f788'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.566077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0cf75c87-377e-4870-8840-5d558f3197b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1110c084e54907a2c4eae2e146f788'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.649882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1136/2294] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:46 2022] DELETE /identity/v3/users/8c1110c084e54907a2c4eae2e146f788 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:46.742320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6540a053913a41f0868923ae10cb6c46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.758456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6540a053913a41f0868923ae10cb6c46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.759325 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6540a053913a41f0868923ae10cb6c46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.760161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6540a053913a41f0868923ae10cb6c46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.767678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6540a053913a41f0868923ae10cb6c46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:46.864539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1159/2295] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:56:46 2022] DELETE /identity/v3/users/6540a053913a41f0868923ae10cb6c46 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:56:46.864882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7b64-eab7-4c37-ad15-839c6eef1a14 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:46.865183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7b64-eab7-4c37-ad15-839c6eef1a14 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:46.865467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb7b64-eab7-4c37-ad15-839c6eef1a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:46.897694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a9eb7b64-eab7-4c37-ad15-839c6eef1a14 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:46.898253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a9eb7b64-eab7-4c37-ad15-839c6eef1a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:46.994088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1137/2296] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:47.043497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:47.043497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:47.095420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6f21-94aa-46c7-8033-8c9ee2a7be3f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:47.099579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661198c-1aca-472a-b7fd-3ee1864532de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:47.100350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661198c-1aca-472a-b7fd-3ee1864532de service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:47.100697 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661198c-1aca-472a-b7fd-3ee1864532de service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:47.101576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661198c-1aca-472a-b7fd-3ee1864532de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:47.111745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661198c-1aca-472a-b7fd-3ee1864532de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:47.113274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661198c-1aca-472a-b7fd-3ee1864532de service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:47.125789 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1160/2297] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:47.241928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb7b64-eab7-4c37-ad15-839c6eef1a14 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:47.243931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a9eb7b64-eab7-4c37-ad15-839c6eef1a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:47.266970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb7b64-eab7-4c37-ad15-839c6eef1a14 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:47.282090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:47.282992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:47.283284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:47.283601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] PATH_INFO: `/v3/projects/15e6b1fa2ab74eaca163b2928d6c85df` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:47.286193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e6b1fa2ab74eaca163b2928d6c85df)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:47.291807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:47.297279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:47.431599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e6b1fa2ab74eaca163b2928d6c85df', 'request_id': 'req-9f769cb3-f785-4dc0-a27b-029fee93edea'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:47.438179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e6b1fa2ab74eaca163b2928d6c85df', 'request_id': 'req-9f769cb3-f785-4dc0-a27b-029fee93edea'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:47.446263 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e6b1fa2ab74eaca163b2928d6c85df', 'request_id': 'req-9f769cb3-f785-4dc0-a27b-029fee93edea'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:47.544503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] The token cache is being invalidate because project 15e6b1fa2ab74eaca163b2928d6c85df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:47.545227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:47.548559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f769cb3-f785-4dc0-a27b-029fee93edea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:47.553526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1138/2298] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:47 2022] DELETE /identity/v3/projects/15e6b1fa2ab74eaca163b2928d6c85df => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:47.559116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ae4ca-e296-46a6-a509-cf00d13d2e19 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:47.559694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ae4ca-e296-46a6-a509-cf00d13d2e19 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:47.572149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ae4ca-e296-46a6-a509-cf00d13d2e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:47.620252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-565ae4ca-e296-46a6-a509-cf00d13d2e19 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:47.621724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-565ae4ca-e296-46a6-a509-cf00d13d2e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:47.719266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:47.736169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1161/2299] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:47.755690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afea9e-77be-4d22-958f-d8559725286b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:47.755985 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afea9e-77be-4d22-958f-d8559725286b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:47.756268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afea9e-77be-4d22-958f-d8559725286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:47.780448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ae4ca-e296-46a6-a509-cf00d13d2e19 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:47.789185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a7afea9e-77be-4d22-958f-d8559725286b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:47.789328 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-565ae4ca-e296-46a6-a509-cf00d13d2e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:47.789448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a7afea9e-77be-4d22-958f-d8559725286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:47.797070 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1139/2300] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:47.798495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afea9e-77be-4d22-958f-d8559725286b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:47.808939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ae4ca-e296-46a6-a509-cf00d13d2e19 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:47.812923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afea9e-77be-4d22-958f-d8559725286b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:47.815576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:47.816801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:47.818515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:47.819002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] PATH_INFO: `/v3/projects/cd50bd3524414dff85babb547c38f2bb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:47.819382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:47.820621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:47.821000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:47.821387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:47.821648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd50bd3524414dff85babb547c38f2bb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:47.822455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:56:47.822906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:47.823952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:47.824958 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:47.847008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1140/2301] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:56:47.848531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:47.869477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c85981-72d0-4030-9ef7-e0f469038b2e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:47.888475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff25c-3416-4801-aff9-9021dff0b325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:47.893473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff25c-3416-4801-aff9-9021dff0b325 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:47.894013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff25c-3416-4801-aff9-9021dff0b325 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:47.894448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff25c-3416-4801-aff9-9021dff0b325 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:47.895893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff25c-3416-4801-aff9-9021dff0b325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:47.902171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff25c-3416-4801-aff9-9021dff0b325 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.004733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd50bd3524414dff85babb547c38f2bb', 'request_id': 'req-b21592e5-6b42-4d61-b054-62be0bddceaa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:48.012147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd50bd3524414dff85babb547c38f2bb', 'request_id': 'req-b21592e5-6b42-4d61-b054-62be0bddceaa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:48.082541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1141/2302] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:47 2022] POST /identity/v3/projects => generated 383 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:48.084088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff25c-3416-4801-aff9-9021dff0b325 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.102358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd50bd3524414dff85babb547c38f2bb', 'request_id': 'req-b21592e5-6b42-4d61-b054-62be0bddceaa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:48.126781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] The token cache is being invalidate because project cd50bd3524414dff85babb547c38f2bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:56:48.127072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:48.129129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b21592e5-6b42-4d61-b054-62be0bddceaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:56:48.131011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1162/2303] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:56:47 2022] DELETE /identity/v3/projects/cd50bd3524414dff85babb547c38f2bb => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:56:48.137468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad11c3-0f2f-4b33-a7b7-e70647574404 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.137832 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad11c3-0f2f-4b33-a7b7-e70647574404 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.138177 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad11c3-0f2f-4b33-a7b7-e70647574404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.159316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-27ad11c3-0f2f-4b33-a7b7-e70647574404 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:48.160741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-27ad11c3-0f2f-4b33-a7b7-e70647574404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:48.218539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff25c-3416-4801-aff9-9021dff0b325 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.228560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.229344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.229725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.230079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.230849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.236720 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.243804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1163/2304] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:48.256962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcacce-c8aa-40f8-adcc-572e4639dfff None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.258021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcacce-c8aa-40f8-adcc-572e4639dfff None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.258933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcacce-c8aa-40f8-adcc-572e4639dfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.280994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7fbcacce-c8aa-40f8-adcc-572e4639dfff None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:48.282804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7fbcacce-c8aa-40f8-adcc-572e4639dfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:48.294858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1164/2305] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:56:48.298996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbcacce-c8aa-40f8-adcc-572e4639dfff None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.316301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbcacce-c8aa-40f8-adcc-572e4639dfff None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.322289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.323424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.323958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.324338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.325395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:56:48.326081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.327875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.371003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1165/2306] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 15:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:56:48.375633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.391243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1142/2307] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:48 2022] POST /identity/v3/users => generated 292 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:48.394190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589798d-cffc-4182-924f-d944a59327c7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.394991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.396914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:48.399627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0cc68-69f7-4dac-ac72-1c051261515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.400292 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0cc68-69f7-4dac-ac72-1c051261515f admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.400576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0cc68-69f7-4dac-ac72-1c051261515f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.400850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0cc68-69f7-4dac-ac72-1c051261515f admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.401537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0cc68-69f7-4dac-ac72-1c051261515f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.409722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0cc68-69f7-4dac-ac72-1c051261515f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.415120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d22cf1-4294-4497-ba5d-c8669f34449c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.421502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669f23c-fde2-4886-9275-4d38cc9144f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.422704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f23c-fde2-4886-9275-4d38cc9144f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.422704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f23c-fde2-4886-9275-4d38cc9144f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.422899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f23c-fde2-4886-9275-4d38cc9144f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.423629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669f23c-fde2-4886-9275-4d38cc9144f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.425277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669f23c-fde2-4886-9275-4d38cc9144f4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.439340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1143/2308] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:56:48.443081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669f23c-fde2-4886-9275-4d38cc9144f4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.453698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669f23c-fde2-4886-9275-4d38cc9144f4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.458546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b384c3-6230-412c-8fb0-44ad909c57a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.459237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b384c3-6230-412c-8fb0-44ad909c57a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.459578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b384c3-6230-412c-8fb0-44ad909c57a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.459866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b384c3-6230-412c-8fb0-44ad909c57a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.461155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b384c3-6230-412c-8fb0-44ad909c57a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.463283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b384c3-6230-412c-8fb0-44ad909c57a3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.476899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1144/2309] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:48.479393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b384c3-6230-412c-8fb0-44ad909c57a3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.491218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b384c3-6230-412c-8fb0-44ad909c57a3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.496835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d8137-9af4-4133-bca2-b2fc529ad09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.497913 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d8137-9af4-4133-bca2-b2fc529ad09f admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.498211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d8137-9af4-4133-bca2-b2fc529ad09f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.498496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d8137-9af4-4133-bca2-b2fc529ad09f admin admin] PATH_INFO: `/v3/projects/68f37be2c6b9448a91db5337cbd980a3/users/62dccc1fdd9c4d43bb51262bb780b907/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.519588 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d8137-9af4-4133-bca2-b2fc529ad09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f37be2c6b9448a91db5337cbd980a3, user_id=62dccc1fdd9c4d43bb51262bb780b907, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.524191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1166/2310] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:48 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:56:48.525639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d8137-9af4-4133-bca2-b2fc529ad09f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.527719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:48.528565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0cc68-69f7-4dac-ac72-1c051261515f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.601967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0cc68-69f7-4dac-ac72-1c051261515f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.607352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e859d-e055-4813-a5e1-6cf94ffef50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.608019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e859d-e055-4813-a5e1-6cf94ffef50b admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.608311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e859d-e055-4813-a5e1-6cf94ffef50b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.609005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e859d-e055-4813-a5e1-6cf94ffef50b admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.609287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e859d-e055-4813-a5e1-6cf94ffef50b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.611279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e859d-e055-4813-a5e1-6cf94ffef50b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.624215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1145/2311] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:48 2022] PUT /identity/v3/projects/68f37be2c6b9448a91db5337cbd980a3/users/62dccc1fdd9c4d43bb51262bb780b907/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:56:48.637360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd779a7-22ac-4ec1-97e9-9240c7532417 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.637360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd779a7-22ac-4ec1-97e9-9240c7532417 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.637675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd779a7-22ac-4ec1-97e9-9240c7532417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.669558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6bd779a7-22ac-4ec1-97e9-9240c7532417 None None] MFA Rules not processed for user `62dccc1fdd9c4d43bb51262bb780b907`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:48.670797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6bd779a7-22ac-4ec1-97e9-9240c7532417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:48.690346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1167/2312] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:48 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:48.694542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e859d-e055-4813-a5e1-6cf94ffef50b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.715388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e859d-e055-4813-a5e1-6cf94ffef50b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.754401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:48.791347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1146/2313] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:48.796148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd779a7-22ac-4ec1-97e9-9240c7532417 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.837845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1974c7f-3a39-478a-a0d7-a4b50bab79f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.838796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974c7f-3a39-478a-a0d7-a4b50bab79f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.839179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974c7f-3a39-478a-a0d7-a4b50bab79f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.839556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974c7f-3a39-478a-a0d7-a4b50bab79f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.840368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1974c7f-3a39-478a-a0d7-a4b50bab79f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.842885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1974c7f-3a39-478a-a0d7-a4b50bab79f4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.883254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1168/2314] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:48.886449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1974c7f-3a39-478a-a0d7-a4b50bab79f4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.902237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1974c7f-3a39-478a-a0d7-a4b50bab79f4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.902951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd779a7-22ac-4ec1-97e9-9240c7532417 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.912076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e9a36-35a9-4bd0-a933-6ac2514ec5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.913225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e9a36-35a9-4bd0-a933-6ac2514ec5ff admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.913690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e9a36-35a9-4bd0-a933-6ac2514ec5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.914122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e9a36-35a9-4bd0-a933-6ac2514ec5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.915058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e9a36-35a9-4bd0-a933-6ac2514ec5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.916724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504d425-c6da-4b2a-b342-afcddd714718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.917420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504d425-c6da-4b2a-b342-afcddd714718 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.917717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504d425-c6da-4b2a-b342-afcddd714718 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.917832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e9a36-35a9-4bd0-a933-6ac2514ec5ff admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.918001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504d425-c6da-4b2a-b342-afcddd714718 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:48.918734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504d425-c6da-4b2a-b342-afcddd714718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:48.920449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504d425-c6da-4b2a-b342-afcddd714718 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:48.946922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1169/2315] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:56:48.952161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504d425-c6da-4b2a-b342-afcddd714718 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:48.972729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504d425-c6da-4b2a-b342-afcddd714718 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:48.974613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f6ac4-14bf-4eaa-817f-9352e8cdbaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:48.975939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6ac4-14bf-4eaa-817f-9352e8cdbaa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:48.976449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6ac4-14bf-4eaa-817f-9352e8cdbaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:48.976906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f6ac4-14bf-4eaa-817f-9352e8cdbaa8 admin admin] PATH_INFO: `/v3/projects/3f2dce4132ee48c7879319206fcee635/users/a9e02c61bdce42ab9ffff539b231ecc7/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.001399 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f6ac4-14bf-4eaa-817f-9352e8cdbaa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2dce4132ee48c7879319206fcee635, user_id=a9e02c61bdce42ab9ffff539b231ecc7, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:49.005020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f6ac4-14bf-4eaa-817f-9352e8cdbaa8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:49.044316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1147/2316] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 15:56:48 2022] POST /identity/v3/projects => generated 385 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:56:49.046038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e9a36-35a9-4bd0-a933-6ac2514ec5ff admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:49.127099 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e9a36-35a9-4bd0-a933-6ac2514ec5ff admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:49.132717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae69326-d4af-4745-894e-aaf80b00be8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:49.133398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae69326-d4af-4745-894e-aaf80b00be8b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.133837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae69326-d4af-4745-894e-aaf80b00be8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.134133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae69326-d4af-4745-894e-aaf80b00be8b admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.134847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae69326-d4af-4745-894e-aaf80b00be8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:49.136888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae69326-d4af-4745-894e-aaf80b00be8b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:49.139818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1170/2317] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:48 2022] PUT /identity/v3/projects/3f2dce4132ee48c7879319206fcee635/users/a9e02c61bdce42ab9ffff539b231ecc7/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:56:49.152241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc0ad9-b7cf-41f4-b6b3-f012e6498795 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.152818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc0ad9-b7cf-41f4-b6b3-f012e6498795 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.153203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc0ad9-b7cf-41f4-b6b3-f012e6498795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.199713 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3cfc0ad9-b7cf-41f4-b6b3-f012e6498795 None None] MFA Rules not processed for user `a9e02c61bdce42ab9ffff539b231ecc7`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:49.201022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3cfc0ad9-b7cf-41f4-b6b3-f012e6498795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:49.242394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1148/2318] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 15:56:49 2022] POST /identity/v3/users => generated 299 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:56:49.246253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae69326-d4af-4745-894e-aaf80b00be8b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:49.263723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae69326-d4af-4745-894e-aaf80b00be8b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:49.319122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1171/2319] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:49.338782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc0ad9-b7cf-41f4-b6b3-f012e6498795 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:49.340692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3cfc0ad9-b7cf-41f4-b6b3-f012e6498795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:49.352024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef961-8719-4a12-a7ef-2b181cad6fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:49.352737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef961-8719-4a12-a7ef-2b181cad6fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.352988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef961-8719-4a12-a7ef-2b181cad6fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.353308 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef961-8719-4a12-a7ef-2b181cad6fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.355199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ef961-8719-4a12-a7ef-2b181cad6fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:49.357264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ef961-8719-4a12-a7ef-2b181cad6fa8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:49.379247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1149/2320] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:56:49.380351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef961-8719-4a12-a7ef-2b181cad6fa8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:49.405728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef961-8719-4a12-a7ef-2b181cad6fa8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:49.419454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1bad5-d3a1-40ee-a802-6d67f6010f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:49.423596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1bad5-d3a1-40ee-a802-6d67f6010f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.423596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1bad5-d3a1-40ee-a802-6d67f6010f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.423596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1bad5-d3a1-40ee-a802-6d67f6010f66 admin admin] PATH_INFO: `/v3/projects/39143dd410874ba0a850552c8980b2dc/users/d2e314ad75f74179ba0e13510252e455/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.448497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1bad5-d3a1-40ee-a802-6d67f6010f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39143dd410874ba0a850552c8980b2dc, user_id=d2e314ad75f74179ba0e13510252e455, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:49.452402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc0ad9-b7cf-41f4-b6b3-f012e6498795 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:49.452856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1bad5-d3a1-40ee-a802-6d67f6010f66 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:49.457248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5922a-2dfc-4136-b7a2-2ed060d00eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:49.457920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5922a-2dfc-4136-b7a2-2ed060d00eef service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.458267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5922a-2dfc-4136-b7a2-2ed060d00eef service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.458550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5922a-2dfc-4136-b7a2-2ed060d00eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.470472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5922a-2dfc-4136-b7a2-2ed060d00eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:49.472232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5922a-2dfc-4136-b7a2-2ed060d00eef service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:49.482802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1172/2321] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:56:49.558493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1150/2322] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:49 2022] PUT /identity/v3/projects/39143dd410874ba0a850552c8980b2dc/users/d2e314ad75f74179ba0e13510252e455/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:49.563809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5922a-2dfc-4136-b7a2-2ed060d00eef service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:49.578493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5922a-2dfc-4136-b7a2-2ed060d00eef service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:49.656938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:49.659317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a4c47-a34d-40b1-93a1-908b350e29fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:49.660294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4c47-a34d-40b1-93a1-908b350e29fc admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.660586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4c47-a34d-40b1-93a1-908b350e29fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.661850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a4c47-a34d-40b1-93a1-908b350e29fc admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.662663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a4c47-a34d-40b1-93a1-908b350e29fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:49.664900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a4c47-a34d-40b1-93a1-908b350e29fc admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:49.675640 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1173/2323] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:56:49.680471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1bad5-d3a1-40ee-a802-6d67f6010f66 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:49.690968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1bad5-d3a1-40ee-a802-6d67f6010f66 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:49.696351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:49.697369 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.697653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.697927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] PATH_INFO: `/v3/projects/39143dd410874ba0a850552c8980b2dc/users/d2e314ad75f74179ba0e13510252e455/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.699973 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39143dd410874ba0a850552c8980b2dc, user_id=d2e314ad75f74179ba0e13510252e455, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:49.702714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:49.719521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1151/2324] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 15:56:49 2022] PUT /identity/v3/projects/39143dd410874ba0a850552c8980b2dc/users/d2e314ad75f74179ba0e13510252e455/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:56:49.733318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae5f81-f33a-44cd-b863-8292bc82789d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.734357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae5f81-f33a-44cd-b863-8292bc82789d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.734357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae5f81-f33a-44cd-b863-8292bc82789d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.768138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-63ae5f81-f33a-44cd-b863-8292bc82789d None None] MFA Rules not processed for user `d2e314ad75f74179ba0e13510252e455`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:49.769768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-63ae5f81-f33a-44cd-b863-8292bc82789d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:49.841743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1174/2325] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:49.853205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:49.861331 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:49.925052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590d57b-a31a-4fc1-8dcc-6bfee6c4388f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:49.928626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:49.929361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:49.929647 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:49.929935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:49.971813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:49.993357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:49.994862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:50.009306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1152/2326] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:52.432087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a3ea4-5bb0-4c39-baa5-b23881515ee1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:52.432626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a3ea4-5bb0-4c39-baa5-b23881515ee1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:52.432626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a3ea4-5bb0-4c39-baa5-b23881515ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:52.450495 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-726a3ea4-5bb0-4c39-baa5-b23881515ee1 None None] MFA Rules not processed for user `a9e02c61bdce42ab9ffff539b231ecc7`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:56:52.451740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-726a3ea4-5bb0-4c39-baa5-b23881515ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:52.487340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:52.509016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1175/2327] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:56:55.093495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:55.095517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:55.110102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:56:55.167997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc84188-8a6d-41ed-8f60-0dc878892c97 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:55.173119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:55.181327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:55.181327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:55.181327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:55.181327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:56:55.188665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:55.190262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:55.200312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1153/2328] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:56:59.085134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a3ea4-5bb0-4c39-baa5-b23881515ee1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:56:59.095555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a3ea4-5bb0-4c39-baa5-b23881515ee1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:56:59.099718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a568fa-11ef-4e97-bdd1-ed9f72785e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:56:59.100459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a568fa-11ef-4e97-bdd1-ed9f72785e5b service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:56:59.100745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a568fa-11ef-4e97-bdd1-ed9f72785e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:56:59.101238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a568fa-11ef-4e97-bdd1-ed9f72785e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:56:59.113930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a568fa-11ef-4e97-bdd1-ed9f72785e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:56:59.115450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a568fa-11ef-4e97-bdd1-ed9f72785e5b service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:56:59.125861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1176/2329] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:56:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:57:05.447629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=27 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:57:06.750074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:06.814585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccd409-5698-4a97-afbf-d7d662dfe0a9 service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:06.819133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:06.820041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:06.820333 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:06.820644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] PATH_INFO: `/v3/users/62dccc1fdd9c4d43bb51262bb780b907` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:06.821879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62dccc1fdd9c4d43bb51262bb780b907)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:06.824012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:07.089445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62dccc1fdd9c4d43bb51262bb780b907'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:07.102192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62dccc1fdd9c4d43bb51262bb780b907'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:07.103019 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62dccc1fdd9c4d43bb51262bb780b907'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:07.103694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62dccc1fdd9c4d43bb51262bb780b907'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:07.114853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62dccc1fdd9c4d43bb51262bb780b907'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:07.215546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1154/2330] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:57:06 2022] DELETE /identity/v3/users/62dccc1fdd9c4d43bb51262bb780b907 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:57:07.217576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfed5ba-d200-4fae-92f2-273f43cd6c03 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:07.218061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfed5ba-d200-4fae-92f2-273f43cd6c03 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:07.218176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfed5ba-d200-4fae-92f2-273f43cd6c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:07.243964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1dfed5ba-d200-4fae-92f2-273f43cd6c03 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:07.245527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1dfed5ba-d200-4fae-92f2-273f43cd6c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:07.348574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1177/2331] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:57:07.359198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:07.360813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:07.429020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975538bb-c115-4269-8dee-62f856140611 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:07.432752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:07.433447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:07.433682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:07.433965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:07.443658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:07.445657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:07.455443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1155/2332] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:57:07.514191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfed5ba-d200-4fae-92f2-273f43cd6c03 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:07.515642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1dfed5ba-d200-4fae-92f2-273f43cd6c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:07.528165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfed5ba-d200-4fae-92f2-273f43cd6c03 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:07.537874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:07.538142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:07.538610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:07.539039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:07.541434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:07.605943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:07.607344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:07.618128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1178/2333] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:57:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:57:07.887145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:07.888782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:07.909472 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612c256-02d3-4f77-9005-4fe987ae459d service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:07.914937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:07.915937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:07.916416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:07.916792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] PATH_INFO: `/v3/projects/68f37be2c6b9448a91db5337cbd980a3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:07.918216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f37be2c6b9448a91db5337cbd980a3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:07.921064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:07.941064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:57:08.055013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f37be2c6b9448a91db5337cbd980a3', 'request_id': 'req-9deb563a-0415-4628-9308-2016c1086237'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.063133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f37be2c6b9448a91db5337cbd980a3', 'request_id': 'req-9deb563a-0415-4628-9308-2016c1086237'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.073475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f37be2c6b9448a91db5337cbd980a3', 'request_id': 'req-9deb563a-0415-4628-9308-2016c1086237'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.188862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] The token cache is being invalidate because project 68f37be2c6b9448a91db5337cbd980a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:57:08.190183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.190183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.191746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1156/2334] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:57:07 2022] DELETE /identity/v3/projects/68f37be2c6b9448a91db5337cbd980a3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:57:08.195693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:08.206960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7070d-eafe-49de-813b-0ca1dcb178bb service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:08.265113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:08.266002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:08.266291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:08.266565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] PATH_INFO: `/v3/users/d2e314ad75f74179ba0e13510252e455` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:08.267787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e314ad75f74179ba0e13510252e455)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:08.269740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:08.276366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:57:08.469024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e314ad75f74179ba0e13510252e455'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.481644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e314ad75f74179ba0e13510252e455'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.482396 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e314ad75f74179ba0e13510252e455'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.483004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e314ad75f74179ba0e13510252e455'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.491280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86563cb5-81fd-4056-98db-573070476061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e314ad75f74179ba0e13510252e455'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:08.571998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1179/2335] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:57:08 2022] DELETE /identity/v3/users/d2e314ad75f74179ba0e13510252e455 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:57:09.424294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:09.426075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:09.442086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb563a-0415-4628-9308-2016c1086237 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:09.510445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:09.511382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:09.511676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:09.511955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] PATH_INFO: `/v3/projects/39143dd410874ba0a850552c8980b2dc` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:09.513210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39143dd410874ba0a850552c8980b2dc)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:09.515208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:09.628624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39143dd410874ba0a850552c8980b2dc', 'request_id': 'req-7d3b91ef-cee7-4322-8c6d-37af1804eb96'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:09.635846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39143dd410874ba0a850552c8980b2dc', 'request_id': 'req-7d3b91ef-cee7-4322-8c6d-37af1804eb96'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:09.641732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39143dd410874ba0a850552c8980b2dc', 'request_id': 'req-7d3b91ef-cee7-4322-8c6d-37af1804eb96'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:09.763882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] The token cache is being invalidate because project 39143dd410874ba0a850552c8980b2dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:57:09.768220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:09.768220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:09.768220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1157/2336] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:57:09 2022] DELETE /identity/v3/projects/39143dd410874ba0a850552c8980b2dc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:57:16.658428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29548965-9094-42db-bb01-d16651a7ff16 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:16.658927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29548965-9094-42db-bb01-d16651a7ff16 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:16.659024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29548965-9094-42db-bb01-d16651a7ff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:16.680074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-29548965-9094-42db-bb01-d16651a7ff16 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:16.681314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-29548965-9094-42db-bb01-d16651a7ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:16.743505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1180/2337] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:16.769682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:16.771244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:16.813766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:57:16.843784 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b91ef-cee7-4322-8c6d-37af1804eb96 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:16.850176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057645c-62ef-478d-af56-e6c0b6d826d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:16.850837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057645c-62ef-478d-af56-e6c0b6d826d9 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:16.851190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057645c-62ef-478d-af56-e6c0b6d826d9 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:16.858666 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057645c-62ef-478d-af56-e6c0b6d826d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:16.860314 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057645c-62ef-478d-af56-e6c0b6d826d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:16.861908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057645c-62ef-478d-af56-e6c0b6d826d9 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:16.870478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1158/2338] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:57:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:57:20.496349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a09b00-1593-48be-ab02-4e5265e164ae None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:20.499714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a09b00-1593-48be-ab02-4e5265e164ae None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:20.499714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a09b00-1593-48be-ab02-4e5265e164ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:20.517730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-03a09b00-1593-48be-ab02-4e5265e164ae None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:20.518995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-03a09b00-1593-48be-ab02-4e5265e164ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:20.527425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1181/2339] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:20.540051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057645c-62ef-478d-af56-e6c0b6d826d9 service swift] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:20.550444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057645c-62ef-478d-af56-e6c0b6d826d9 service swift] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:20.555547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc269a9-0672-4778-9389-993ae6d1d130 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:20.556176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc269a9-0672-4778-9389-993ae6d1d130 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:20.556459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc269a9-0672-4778-9389-993ae6d1d130 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:20.556739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc269a9-0672-4778-9389-993ae6d1d130 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:20.565182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc269a9-0672-4778-9389-993ae6d1d130 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:20.566765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc269a9-0672-4778-9389-993ae6d1d130 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:20.577145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1159/2340] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:57:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:57:21.817906 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aca8e3-4fc7-4944-b145-4834cbe82e83 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:21.818438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aca8e3-4fc7-4944-b145-4834cbe82e83 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:21.818532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aca8e3-4fc7-4944-b145-4834cbe82e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:21.840685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-92aca8e3-4fc7-4944-b145-4834cbe82e83 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:21.842443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-92aca8e3-4fc7-4944-b145-4834cbe82e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:21.861338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1182/2341] 10.222.0.98 () {58 vars in 967 bytes} [Fri Jul 15 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:21.878384 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc269a9-0672-4778-9389-993ae6d1d130 service swift] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:21.892396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc269a9-0672-4778-9389-993ae6d1d130 service swift] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:21.900113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78358462-6a9b-402d-818f-c2400eb908b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:21.903972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78358462-6a9b-402d-818f-c2400eb908b9 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:21.903972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78358462-6a9b-402d-818f-c2400eb908b9 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:21.903972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78358462-6a9b-402d-818f-c2400eb908b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:21.915200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78358462-6a9b-402d-818f-c2400eb908b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:21.917243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78358462-6a9b-402d-818f-c2400eb908b9 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:21.927015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1160/2342] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:57:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:57:22.621950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95026ee-cf62-4224-a7c0-b1bd021a5680 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:22.622184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95026ee-cf62-4224-a7c0-b1bd021a5680 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:22.622649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95026ee-cf62-4224-a7c0-b1bd021a5680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:22.642870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b95026ee-cf62-4224-a7c0-b1bd021a5680 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:22.644316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b95026ee-cf62-4224-a7c0-b1bd021a5680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:22.653290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1183/2343] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:22.671815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78358462-6a9b-402d-818f-c2400eb908b9 service swift] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:22.685654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78358462-6a9b-402d-818f-c2400eb908b9 service swift] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:22.694503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5563c96-4b22-4fed-8089-02f9357730ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:22.695021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563c96-4b22-4fed-8089-02f9357730ed service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:22.695457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563c96-4b22-4fed-8089-02f9357730ed service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:22.695915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5563c96-4b22-4fed-8089-02f9357730ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:22.705613 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5563c96-4b22-4fed-8089-02f9357730ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:22.707312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5563c96-4b22-4fed-8089-02f9357730ed service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:22.717291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1161/2344] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:57:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:57:23.434159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbd3e9-51c0-459f-8b7d-01e84f9cc3c8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:23.434621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbd3e9-51c0-459f-8b7d-01e84f9cc3c8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:23.434724 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbd3e9-51c0-459f-8b7d-01e84f9cc3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:23.452853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a2dbd3e9-51c0-459f-8b7d-01e84f9cc3c8 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:23.454138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a2dbd3e9-51c0-459f-8b7d-01e84f9cc3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:23.462321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1184/2345] 10.222.0.98 () {58 vars in 967 bytes} [Fri Jul 15 15:57:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:23.474831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5563c96-4b22-4fed-8089-02f9357730ed service swift] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:23.487343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5563c96-4b22-4fed-8089-02f9357730ed service swift] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:23.491676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcef8d7-9729-4674-8cbb-fdb8dc11819a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:23.492334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcef8d7-9729-4674-8cbb-fdb8dc11819a service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:23.492621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcef8d7-9729-4674-8cbb-fdb8dc11819a service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:23.492902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcef8d7-9729-4674-8cbb-fdb8dc11819a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:23.501845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcef8d7-9729-4674-8cbb-fdb8dc11819a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:23.503492 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcef8d7-9729-4674-8cbb-fdb8dc11819a service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:23.512183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1162/2346] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:57:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:57:26.776847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbd3e9-51c0-459f-8b7d-01e84f9cc3c8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:26.776847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a2dbd3e9-51c0-459f-8b7d-01e84f9cc3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:26.842158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbd3e9-51c0-459f-8b7d-01e84f9cc3c8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:26.846783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:26.847694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:26.847984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:26.848264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] PATH_INFO: `/v3/users/a9e02c61bdce42ab9ffff539b231ecc7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:26.849473 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e02c61bdce42ab9ffff539b231ecc7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:26.851489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:26.856048 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:57:27.101722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e02c61bdce42ab9ffff539b231ecc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:27.113214 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e02c61bdce42ab9ffff539b231ecc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:27.113879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e02c61bdce42ab9ffff539b231ecc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:27.115015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e02c61bdce42ab9ffff539b231ecc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:27.121952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e02c61bdce42ab9ffff539b231ecc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:27.245897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1185/2347] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 15:57:26 2022] DELETE /identity/v3/users/a9e02c61bdce42ab9ffff539b231ecc7 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:57:27.250691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36b138-7735-4eb6-82b4-492e3f9786e6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:27.251175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36b138-7735-4eb6-82b4-492e3f9786e6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:27.251280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36b138-7735-4eb6-82b4-492e3f9786e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:27.270683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-eb36b138-7735-4eb6-82b4-492e3f9786e6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:27.272379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-eb36b138-7735-4eb6-82b4-492e3f9786e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:27.351437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1163/2348] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:27.358117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:27.359804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:27.432020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10196b-dc1b-47a5-8e78-0f2c457620ff admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:27.435974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd610fe-1cef-4611-8d59-ffcf4d9ee963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:27.436714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd610fe-1cef-4611-8d59-ffcf4d9ee963 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:27.437071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd610fe-1cef-4611-8d59-ffcf4d9ee963 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:27.437419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd610fe-1cef-4611-8d59-ffcf4d9ee963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:27.448529 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd610fe-1cef-4611-8d59-ffcf4d9ee963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:27.450089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd610fe-1cef-4611-8d59-ffcf4d9ee963 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:27.461945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1186/2349] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:57:28.004156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36b138-7735-4eb6-82b4-492e3f9786e6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:28.005587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-eb36b138-7735-4eb6-82b4-492e3f9786e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:28.018120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36b138-7735-4eb6-82b4-492e3f9786e6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:28.023130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:28.024053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:28.024346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:28.026300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] PATH_INFO: `/v3/projects/3f2dce4132ee48c7879319206fcee635` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:28.028579 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2dce4132ee48c7879319206fcee635)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:28.030608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:28.224558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2dce4132ee48c7879319206fcee635', 'request_id': 'req-0ed6f456-6346-4064-ab1a-2f422189f970'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:28.230892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2dce4132ee48c7879319206fcee635', 'request_id': 'req-0ed6f456-6346-4064-ab1a-2f422189f970'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:28.236432 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2dce4132ee48c7879319206fcee635', 'request_id': 'req-0ed6f456-6346-4064-ab1a-2f422189f970'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:28.240728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:57:28.336543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] The token cache is being invalidate because project 3f2dce4132ee48c7879319206fcee635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:57:28.337067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:28.337528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0ed6f456-6346-4064-ab1a-2f422189f970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:57:28.339175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1164/2350] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 15:57:28 2022] DELETE /identity/v3/projects/3f2dce4132ee48c7879319206fcee635 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:57:35.241205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3a1ec-e2c8-4a6a-abd7-6eb26f71c6f6 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:35.242060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3a1ec-e2c8-4a6a-abd7-6eb26f71c6f6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:35.242060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3a1ec-e2c8-4a6a-abd7-6eb26f71c6f6 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:35.243274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1187/2351] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:57:35.249417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0799d06-9971-4a36-b298-913d566dc964 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:35.249889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0799d06-9971-4a36-b298-913d566dc964 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:35.250020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0799d06-9971-4a36-b298-913d566dc964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:35.268563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b0799d06-9971-4a36-b298-913d566dc964 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:35.271080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b0799d06-9971-4a36-b298-913d566dc964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:35.337808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1165/2352] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:57:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:35.349910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3a1ec-e2c8-4a6a-abd7-6eb26f71c6f6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:35.351650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7bf3a1ec-e2c8-4a6a-abd7-6eb26f71c6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:35.408962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3a1ec-e2c8-4a6a-abd7-6eb26f71c6f6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:35.412462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf142833-3e86-44c1-bd58-eb4a582737db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f18d0f5f634d84a99fc3e46963a8b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:35.413120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf142833-3e86-44c1-bd58-eb4a582737db service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:35.413408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf142833-3e86-44c1-bd58-eb4a582737db service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:35.413690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf142833-3e86-44c1-bd58-eb4a582737db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:35.424073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf142833-3e86-44c1-bd58-eb4a582737db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:35.425499 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf142833-3e86-44c1-bd58-eb4a582737db service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:35.436060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1188/2353] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:57:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:57:35.731580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d505e-5dfd-41ac-8e00-c6f16ce01f07 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:35.731913 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d505e-5dfd-41ac-8e00-c6f16ce01f07 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:35.732202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d505e-5dfd-41ac-8e00-c6f16ce01f07 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:35.733606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1166/2354] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:57:35.738604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709d318-5545-4eb0-9cdf-f4e6ec4752c2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:35.738900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709d318-5545-4eb0-9cdf-f4e6ec4752c2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:35.739239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709d318-5545-4eb0-9cdf-f4e6ec4752c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:35.757871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6709d318-5545-4eb0-9cdf-f4e6ec4752c2 None None] MFA Rules not processed for user `9d20de9c921942d1afeb63e87272f782`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:35.759080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6709d318-5545-4eb0-9cdf-f4e6ec4752c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:35.783186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:57:35.809828 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1189/2355] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:57:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:35.822394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d505e-5dfd-41ac-8e00-c6f16ce01f07 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:35.824153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b74d505e-5dfd-41ac-8e00-c6f16ce01f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:35.837533 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d505e-5dfd-41ac-8e00-c6f16ce01f07 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:35.841092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8de06-5996-43e0-a64b-2820d051577c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:35.841730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8de06-5996-43e0-a64b-2820d051577c service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:35.842027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8de06-5996-43e0-a64b-2820d051577c service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:35.846246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8de06-5996-43e0-a64b-2820d051577c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:35.849423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8de06-5996-43e0-a64b-2820d051577c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:35.850870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8de06-5996-43e0-a64b-2820d051577c service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:35.858669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1167/2356] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:57:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:57:35.940605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc11b0a-121b-4c6c-a162-4b35025c82c8 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:35.940900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc11b0a-121b-4c6c-a162-4b35025c82c8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:35.941204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc11b0a-121b-4c6c-a162-4b35025c82c8 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:35.943176 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1190/2357] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 15:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:57:35.948436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ed82c-1a1f-455f-b426-a3d75b6c4c2a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:35.948735 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ed82c-1a1f-455f-b426-a3d75b6c4c2a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:35.949020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ed82c-1a1f-455f-b426-a3d75b6c4c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:35.969398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-958ed82c-1a1f-455f-b426-a3d75b6c4c2a None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:35.970622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-958ed82c-1a1f-455f-b426-a3d75b6c4c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:35.981658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1168/2358] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 15:57:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:35.994199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc11b0a-121b-4c6c-a162-4b35025c82c8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:36.005683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc11b0a-121b-4c6c-a162-4b35025c82c8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:36.009268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:36.009899 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:36.010184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:36.010460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:36.021028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:36.022504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:36.033273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1191/2359] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:57:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:57:40.138558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b879d-253a-4984-ae6a-dc73e800ac04 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:40.139052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b879d-253a-4984-ae6a-dc73e800ac04 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:40.139170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b879d-253a-4984-ae6a-dc73e800ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:40.158811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b21b879d-253a-4984-ae6a-dc73e800ac04 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:57:40.160067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b21b879d-253a-4984-ae6a-dc73e800ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:40.223382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1169/2360] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 15:57:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:57:40.236146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:57:40.237558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:57:40.308413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0c478-f9a7-4f40-9923-3ceae18ceae7 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:57:40.313777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf7139-edd2-4f40-a891-a27cd0ee930e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:57:40.314420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf7139-edd2-4f40-a891-a27cd0ee930e service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:57:40.314702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf7139-edd2-4f40-a891-a27cd0ee930e service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:57:40.314975 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf7139-edd2-4f40-a891-a27cd0ee930e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:57:40.322879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf7139-edd2-4f40-a891-a27cd0ee930e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:57:40.324505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf7139-edd2-4f40-a891-a27cd0ee930e service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:57:40.332330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1192/2361] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:57:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:57:50.213443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=67 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:57:50.329189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=48 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:58:09.760840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826536b0-46a0-42c9-bfdd-73d8e24b21dc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:09.761321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826536b0-46a0-42c9-bfdd-73d8e24b21dc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:09.761413 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826536b0-46a0-42c9-bfdd-73d8e24b21dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:09.778853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-826536b0-46a0-42c9-bfdd-73d8e24b21dc None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:58:09.780080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-826536b0-46a0-42c9-bfdd-73d8e24b21dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:09.787324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1170/2362] 10.222.0.98 () {58 vars in 967 bytes} [Fri Jul 15 15:58:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:58:09.799589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf7139-edd2-4f40-a891-a27cd0ee930e service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:58:09.810773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf7139-edd2-4f40-a891-a27cd0ee930e service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:58:09.816168 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:58:09.816835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:09.817122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:09.817484 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:09.826282 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:58:09.827898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:58:09.836170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1193/2363] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 15:58:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:58:19.774041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:58:19.832773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:58:33.472042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826536b0-46a0-42c9-bfdd-73d8e24b21dc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:58:33.473527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-826536b0-46a0-42c9-bfdd-73d8e24b21dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:33.489623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826536b0-46a0-42c9-bfdd-73d8e24b21dc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:58:33.495139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:58:33.495924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:33.496210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:33.496488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:33.498200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:33.556715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:58:33.558372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:58:33.567673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1171/2364] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:58:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:58:33.619927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:58:33.621429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:33.623672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:58:33.625034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:33.681022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd677-8ec1-47e6-a2ad-c4202a896202 service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:58:33.683933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f29773-7d65-4024-9436-4bcf0a3c3aa1 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:58:33.684679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:58:33.685353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:33.685639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:33.685918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:33.687596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:58:33.687760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:33.688241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:33.688523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:33.688799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:33.690603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:33.701726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:58:33.703153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:58:33.704400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:58:33.705822 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:58:33.715076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1194/2365] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:58:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:58:33.717488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1172/2366] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:58:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:58:41.225411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:58:41.236872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cebc4b-2f06-4908-bd40-4fd28260d26c service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:58:41.242608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107e546-3f79-44f0-bcda-767f036c1dfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:58:41.243243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e546-3f79-44f0-bcda-767f036c1dfc service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:41.244061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e546-3f79-44f0-bcda-767f036c1dfc service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:41.244341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e546-3f79-44f0-bcda-767f036c1dfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:41.246130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c107e546-3f79-44f0-bcda-767f036c1dfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:41.302801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107e546-3f79-44f0-bcda-767f036c1dfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:58:41.304435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107e546-3f79-44f0-bcda-767f036c1dfc service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:58:41.314879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1195/2367] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:58:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:58:43.713072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=54 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:58:43.935758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:58:43.945944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170fa7-2a17-4925-9758-a4234ead49e5 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:58:43.950145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:58:43.950792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:43.951083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:43.951403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:43.953159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:43.963716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:58:43.965097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:58:43.973162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1173/2368] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:58:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:58:51.310650 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=54 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:58:53.156476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c64e-6f72-46ad-8116-a16b8128a81f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:53.157081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c64e-6f72-46ad-8116-a16b8128a81f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:53.157081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2c64e-6f72-46ad-8116-a16b8128a81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:53.178233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fde2c64e-6f72-46ad-8116-a16b8128a81f None None] MFA Rules not processed for user `1157b91a306540dd98a072b1ea4941b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:58:53.179512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fde2c64e-6f72-46ad-8116-a16b8128a81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:53.248039 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1196/2369] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:58:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:58:53.554497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:58:53.563689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101a34-226b-47f1-aa4b-90c83ccb207d service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:58:53.567259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774d141-71c2-43df-a9b1-e94c367ca86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:58:53.567917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d141-71c2-43df-a9b1-e94c367ca86f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:53.568202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d141-71c2-43df-a9b1-e94c367ca86f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:53.568479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774d141-71c2-43df-a9b1-e94c367ca86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:53.570229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9774d141-71c2-43df-a9b1-e94c367ca86f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:53.580580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d141-71c2-43df-a9b1-e94c367ca86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:58:53.582060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774d141-71c2-43df-a9b1-e94c367ca86f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:58:53.590867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1174/2370] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:58:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:58:53.987041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2c64e-6f72-46ad-8116-a16b8128a81f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:58:53.988602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fde2c64e-6f72-46ad-8116-a16b8128a81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:54.072229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2c64e-6f72-46ad-8116-a16b8128a81f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:58:54.077958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:58:54.078616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:58:54.078901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:58:54.079180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:58:54.080976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:58:54.093513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:58:54.095099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:58:54.106186 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1197/2371] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:58:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:59:00.320012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab211f8-029c-473c-86b4-c177ade2f843 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:59:00.320597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab211f8-029c-473c-86b4-c177ade2f843 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:59:00.320597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab211f8-029c-473c-86b4-c177ade2f843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:59:00.338623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4ab211f8-029c-473c-86b4-c177ade2f843 None None] MFA Rules not processed for user `9005cb23317a4a05b58e99284824135e`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:59:00.339940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4ab211f8-029c-473c-86b4-c177ade2f843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:00.349107 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1175/2372] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:59:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:59:04.100855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=57 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:59:10.332597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=15 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:59:11.553249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:59:11.565694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf71e1-c4ce-47df-801e-1f17a5aeacc3 service placement] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:59:11.571414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:59:11.572590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:59:11.572875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:59:11.573154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:59:11.574970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:11.587751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:59:11.589292 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:59:11.599771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1198/2373] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:59:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:59:19.010554 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab211f8-029c-473c-86b4-c177ade2f843 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:59:19.024938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab211f8-029c-473c-86b4-c177ade2f843 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:59:19.030407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb724-cca2-4f09-b759-b4015262477b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:59:19.031077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb724-cca2-4f09-b759-b4015262477b service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:59:19.031492 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb724-cca2-4f09-b759-b4015262477b service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:59:19.031778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb724-cca2-4f09-b759-b4015262477b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:59:19.033597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-791bb724-cca2-4f09-b759-b4015262477b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:19.090563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bb724-cca2-4f09-b759-b4015262477b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:59:19.092190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bb724-cca2-4f09-b759-b4015262477b service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:59:19.101024 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1176/2374] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:59:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:59:21.595352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:59:25.685706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:59:25.690326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:25.702804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c38a80-883f-4b9e-8609-f7fb77e0c101 service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:59:25.708391 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:59:25.709040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:59:25.709327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:59:25.709605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:59:25.711783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:25.725026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:59:25.726688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:59:25.739193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1199/2375] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:59:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:59:29.096892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=28 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:59:30.583061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3586-0f7c-4868-a458-df76f64926a5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:59:30.591477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3586-0f7c-4868-a458-df76f64926a5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:59:30.591477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c3586-0f7c-4868-a458-df76f64926a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:59:30.600653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ec2c3586-0f7c-4868-a458-df76f64926a5 None None] MFA Rules not processed for user `ac8dfa5c97594e74a6839d91b292adcd`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:59:30.601879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ec2c3586-0f7c-4868-a458-df76f64926a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:30.609368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1177/2376] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 15:59:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:59:35.734933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:59:40.595783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:59:48.920363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:59:48.931660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535344a9-9dc5-49e4-ba3a-1b92d234afdf service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:59:48.937902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:59:48.938657 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:59:48.938931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:59:48.942518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:59:48.942518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:48.953806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:59:48.955480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:59:48.966322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1200/2377] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 15:59:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:59:58.961891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:21.591889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532ab12-eb7d-4e9c-80ea-0980021876ee None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:21.593538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532ab12-eb7d-4e9c-80ea-0980021876ee None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:21.593538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532ab12-eb7d-4e9c-80ea-0980021876ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:21.610519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2532ab12-eb7d-4e9c-80ea-0980021876ee None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:00:21.611755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2532ab12-eb7d-4e9c-80ea-0980021876ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:21.618960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1178/2378] 10.222.0.98 () {58 vars in 967 bytes} [Fri Jul 15 16:00:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:00:21.807455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:21.819631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1d549-ffde-4d67-85bf-9148e227d5f9 service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:21.824708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8e616-633d-4cd9-b538-1e0f9a3d41f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:21.825346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8e616-633d-4cd9-b538-1e0f9a3d41f2 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:21.825636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8e616-633d-4cd9-b538-1e0f9a3d41f2 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:21.825916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8e616-633d-4cd9-b538-1e0f9a3d41f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:21.834569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8e616-633d-4cd9-b538-1e0f9a3d41f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:21.836426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8e616-633d-4cd9-b538-1e0f9a3d41f2 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:21.844515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1201/2379] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 16:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 16:00:30.229899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5b08c-2f30-49a7-a510-011c70a94145 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:30.230445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5b08c-2f30-49a7-a510-011c70a94145 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:30.230669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5b08c-2f30-49a7-a510-011c70a94145 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:30.232190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1179/2380] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 16:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 16:00:30.245087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fffa0-656b-4c1c-99cb-1f6cb93d1ec1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:30.245590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fffa0-656b-4c1c-99cb-1f6cb93d1ec1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:30.245935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fffa0-656b-4c1c-99cb-1f6cb93d1ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:30.270403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-051fffa0-656b-4c1c-99cb-1f6cb93d1ec1 None None] MFA Rules not processed for user `e57e2b49a8624266b12c78a78059cc80`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:00:30.272082 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-051fffa0-656b-4c1c-99cb-1f6cb93d1ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:30.282587 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1202/2381] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 16:00:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:00:30.307342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5b08c-2f30-49a7-a510-011c70a94145 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:30.327243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5b08c-2f30-49a7-a510-011c70a94145 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:30.333486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa0d1-92d4-4f1a-83c7-8daaedbb9449 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:30.334475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa0d1-92d4-4f1a-83c7-8daaedbb9449 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:30.334937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa0d1-92d4-4f1a-83c7-8daaedbb9449 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:30.335678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa0d1-92d4-4f1a-83c7-8daaedbb9449 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:30.346367 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daa0d1-92d4-4f1a-83c7-8daaedbb9449 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:30.348033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daa0d1-92d4-4f1a-83c7-8daaedbb9449 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:30.358233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1180/2382] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:00:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:00:30.418094 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fffa0-656b-4c1c-99cb-1f6cb93d1ec1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:30.419733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-051fffa0-656b-4c1c-99cb-1f6cb93d1ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:30.493536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fffa0-656b-4c1c-99cb-1f6cb93d1ec1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:30.498294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:30.499147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:30.500193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:30.500480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] PATH_INFO: `/v3/users/bff41a6af0ea495a856fd5603e810d6a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:30.502143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff41a6af0ea495a856fd5603e810d6a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:30.504154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:30.711999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff41a6af0ea495a856fd5603e810d6a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:30.728626 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff41a6af0ea495a856fd5603e810d6a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:30.729481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff41a6af0ea495a856fd5603e810d6a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:30.732609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff41a6af0ea495a856fd5603e810d6a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:30.743559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff41a6af0ea495a856fd5603e810d6a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:30.849384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1203/2383] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:00:30 2022] DELETE /identity/v3/users/bff41a6af0ea495a856fd5603e810d6a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:00:31.132511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa0d1-92d4-4f1a-83c7-8daaedbb9449 service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:31.167031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa0d1-92d4-4f1a-83c7-8daaedbb9449 service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:31.242389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:31.242389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:31.243087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:31.243196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] PATH_INFO: `/v3/projects/06d54c4865ff45e59526d477accb6cc4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:31.244618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d54c4865ff45e59526d477accb6cc4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:31.246971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:31.397616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d54c4865ff45e59526d477accb6cc4', 'request_id': 'req-17799eeb-3ea4-4cef-939c-ea72c505021d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.409965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d54c4865ff45e59526d477accb6cc4', 'request_id': 'req-17799eeb-3ea4-4cef-939c-ea72c505021d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.418522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d54c4865ff45e59526d477accb6cc4', 'request_id': 'req-17799eeb-3ea4-4cef-939c-ea72c505021d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.494772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] The token cache is being invalidate because project 06d54c4865ff45e59526d477accb6cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:00:31.495211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.495889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.497589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1181/2384] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:00:31 2022] DELETE /identity/v3/projects/06d54c4865ff45e59526d477accb6cc4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:00:31.502238 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:31.504515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:31.519102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bc9b8-da2a-488e-a5f6-58fd57d3051a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:31.578435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:31.579363 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:31.579678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:31.579933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] PATH_INFO: `/v3/users/386071149f824b5d94dfa0e27249f3a3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:31.581131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=386071149f824b5d94dfa0e27249f3a3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:31.583100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:31.584967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:31.789582 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '386071149f824b5d94dfa0e27249f3a3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.804331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '386071149f824b5d94dfa0e27249f3a3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.805029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '386071149f824b5d94dfa0e27249f3a3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.805649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '386071149f824b5d94dfa0e27249f3a3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.813988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cb10f6f7-d228-4241-8415-399ba36c3348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '386071149f824b5d94dfa0e27249f3a3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:31.900792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1204/2385] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:00:31 2022] DELETE /identity/v3/users/386071149f824b5d94dfa0e27249f3a3 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:00:32.142583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:32.144052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:32.157518 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17799eeb-3ea4-4cef-939c-ea72c505021d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:32.216718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:32.217586 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:32.217878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:32.218155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] PATH_INFO: `/v3/projects/d768772b39f840328a0edad241b87b95` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:32.219367 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d768772b39f840328a0edad241b87b95)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:32.221300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:32.232453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:32.371271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd768772b39f840328a0edad241b87b95', 'request_id': 'req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:32.379078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd768772b39f840328a0edad241b87b95', 'request_id': 'req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:32.385746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd768772b39f840328a0edad241b87b95', 'request_id': 'req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:32.499744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] The token cache is being invalidate because project d768772b39f840328a0edad241b87b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:00:32.504941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:32.504941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:32.504941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1182/2386] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:00:32 2022] DELETE /identity/v3/projects/d768772b39f840328a0edad241b87b95 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:00:38.326348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4a63-4d45-40dd-847d-e4a0eac2e60b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:38.326934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4a63-4d45-40dd-847d-e4a0eac2e60b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:38.327032 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a4a63-4d45-40dd-847d-e4a0eac2e60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:38.348386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3d1a4a63-4d45-40dd-847d-e4a0eac2e60b None None] MFA Rules not processed for user `78db558bb3ea4161b4a54e62f9ec3bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:00:38.349649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d1a4a63-4d45-40dd-847d-e4a0eac2e60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:38.417002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1205/2387] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:00:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:00:42.496370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:48.352863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:48.354344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:48.409877 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:48.413916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae584a-47bd-4c27-84c6-bb3df8a57b81 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:48.417744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:48.425313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:48.425313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:48.425313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:48.425313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:48.490099 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:48.492326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:48.502585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1183/2388] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:00:52.552953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a4a63-4d45-40dd-847d-e4a0eac2e60b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:52.554412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d1a4a63-4d45-40dd-847d-e4a0eac2e60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:52.571267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a4a63-4d45-40dd-847d-e4a0eac2e60b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:52.573453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:52.574531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:52.574817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:52.575093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] PATH_INFO: `/v3/users/9005cb23317a4a05b58e99284824135e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:52.576313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9005cb23317a4a05b58e99284824135e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:52.578271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:52.743921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9005cb23317a4a05b58e99284824135e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:52.756805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9005cb23317a4a05b58e99284824135e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:52.757649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9005cb23317a4a05b58e99284824135e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:52.758260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9005cb23317a4a05b58e99284824135e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:52.765761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-79e54578-213b-4981-b752-cbee51af39d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9005cb23317a4a05b58e99284824135e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:52.842539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1206/2389] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:00:52 2022] DELETE /identity/v3/users/9005cb23317a4a05b58e99284824135e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:00:53.078405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:53.090547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b5218-fdd2-415b-9a5a-7799517ef294 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:53.149041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:53.149907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.150223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.150508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] PATH_INFO: `/v3/projects/44cff0ae69e84de486ac2a6866ea6acb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.151730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=44cff0ae69e84de486ac2a6866ea6acb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:53.153703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:53.317229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44cff0ae69e84de486ac2a6866ea6acb', 'request_id': 'req-70c3eb88-6727-4da7-8b6f-91315e640ada'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:53.319193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:53.326780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44cff0ae69e84de486ac2a6866ea6acb', 'request_id': 'req-70c3eb88-6727-4da7-8b6f-91315e640ada'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:53.328961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44cff0ae69e84de486ac2a6866ea6acb', 'request_id': 'req-70c3eb88-6727-4da7-8b6f-91315e640ada'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:53.408568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] The token cache is being invalidate because project 44cff0ae69e84de486ac2a6866ea6acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:00:53.408793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:53.409372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-70c3eb88-6727-4da7-8b6f-91315e640ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:00:53.410908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1184/2390] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:00:53 2022] DELETE /identity/v3/projects/44cff0ae69e84de486ac2a6866ea6acb => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:00:53.415921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4360525-f305-40aa-93e0-dff9aed93805 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.416217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4360525-f305-40aa-93e0-dff9aed93805 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.416501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4360525-f305-40aa-93e0-dff9aed93805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.434631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a4360525-f305-40aa-93e0-dff9aed93805 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:00:53.435962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a4360525-f305-40aa-93e0-dff9aed93805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:53.496263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1207/2391] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:00:53.504448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dfdd7-ff94-4e52-a3c3-19373fd37f66 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.504741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dfdd7-ff94-4e52-a3c3-19373fd37f66 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.505016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dfdd7-ff94-4e52-a3c3-19373fd37f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.524229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a85dfdd7-ff94-4e52-a3c3-19373fd37f66 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:00:53.525463 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a85dfdd7-ff94-4e52-a3c3-19373fd37f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:53.534253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1185/2392] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:00:53.538677 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4360525-f305-40aa-93e0-dff9aed93805 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:53.548812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4360525-f305-40aa-93e0-dff9aed93805 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:53.553543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:53.554220 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.554510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.554792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.555594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:00:53.555915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:53.557584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:53.570424 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1208/2393] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:00:53.574539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dfdd7-ff94-4e52-a3c3-19373fd37f66 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:53.584337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dfdd7-ff94-4e52-a3c3-19373fd37f66 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:53.589273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e6989-1d35-4201-8af6-38b95263519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:53.589939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e6989-1d35-4201-8af6-38b95263519c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.590229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e6989-1d35-4201-8af6-38b95263519c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.590678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e6989-1d35-4201-8af6-38b95263519c admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.591404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e6989-1d35-4201-8af6-38b95263519c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:53.594214 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e6989-1d35-4201-8af6-38b95263519c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:53.719457 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1186/2394] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:00:53 2022] POST /identity/v3/projects => generated 355 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:00:53.723642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:53.734255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28140741-ad0b-486c-8ca2-0a8f422a6fd7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:53.791842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8dd7e-3a58-4cd1-bc39-831cb16fc26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:53.792514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8dd7e-3a58-4cd1-bc39-831cb16fc26c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.792805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8dd7e-3a58-4cd1-bc39-831cb16fc26c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.793088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8dd7e-3a58-4cd1-bc39-831cb16fc26c admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.793784 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8dd7e-3a58-4cd1-bc39-831cb16fc26c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:53.795891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb8dd7e-3a58-4cd1-bc39-831cb16fc26c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:53.802386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:53.860210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1209/2395] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:00:53 2022] POST /identity/v3/users => generated 278 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:00:53.864542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e6989-1d35-4201-8af6-38b95263519c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:53.876114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e6989-1d35-4201-8af6-38b95263519c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:53.880792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae36b9-1883-41e5-b017-c8adcdecd22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:53.881487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae36b9-1883-41e5-b017-c8adcdecd22d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.881775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae36b9-1883-41e5-b017-c8adcdecd22d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.882056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae36b9-1883-41e5-b017-c8adcdecd22d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.882782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae36b9-1883-41e5-b017-c8adcdecd22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:53.884428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae36b9-1883-41e5-b017-c8adcdecd22d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:53.895688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1187/2396] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:00:53.899635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8dd7e-3a58-4cd1-bc39-831cb16fc26c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:53.909616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8dd7e-3a58-4cd1-bc39-831cb16fc26c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:53.914198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49dd16-6adb-4361-8438-f4809e671451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:53.914849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49dd16-6adb-4361-8438-f4809e671451 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.915134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49dd16-6adb-4361-8438-f4809e671451 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.915453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49dd16-6adb-4361-8438-f4809e671451 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.916202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49dd16-6adb-4361-8438-f4809e671451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:53.918112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49dd16-6adb-4361-8438-f4809e671451 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:53.928301 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1210/2397] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:00:53.932371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae36b9-1883-41e5-b017-c8adcdecd22d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:53.942504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae36b9-1883-41e5-b017-c8adcdecd22d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:53.947029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e2f80-ae5f-4875-8534-a781b74aabf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:53.947915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e2f80-ae5f-4875-8534-a781b74aabf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:53.948201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e2f80-ae5f-4875-8534-a781b74aabf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:53.948476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e2f80-ae5f-4875-8534-a781b74aabf7 admin admin] PATH_INFO: `/v3/projects/cc06bcf04d47469fb9a210cb2fbd54e5/users/09b367e11986444c83f600c56b511233/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:53.963274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e2f80-ae5f-4875-8534-a781b74aabf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc06bcf04d47469fb9a210cb2fbd54e5, user_id=09b367e11986444c83f600c56b511233, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:53.965972 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e2f80-ae5f-4875-8534-a781b74aabf7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:54.071058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1188/2398] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:00:53 2022] PUT /identity/v3/projects/cc06bcf04d47469fb9a210cb2fbd54e5/users/09b367e11986444c83f600c56b511233/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:00:54.084426 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335450-1346-4049-af3a-cadf6a207b43 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:54.084723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335450-1346-4049-af3a-cadf6a207b43 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:54.085006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335450-1346-4049-af3a-cadf6a207b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:54.113412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-65335450-1346-4049-af3a-cadf6a207b43 None None] MFA Rules not processed for user `09b367e11986444c83f600c56b511233`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:00:54.114569 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-65335450-1346-4049-af3a-cadf6a207b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:54.183683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1211/2399] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:00:54.188202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e2f80-ae5f-4875-8534-a781b74aabf7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:54.250444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e2f80-ae5f-4875-8534-a781b74aabf7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:54.255098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78663a3f-9304-412c-88db-28aa99e7c921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:54.255781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78663a3f-9304-412c-88db-28aa99e7c921 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:54.256069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78663a3f-9304-412c-88db-28aa99e7c921 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:54.256344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78663a3f-9304-412c-88db-28aa99e7c921 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:54.257020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78663a3f-9304-412c-88db-28aa99e7c921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:54.258920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78663a3f-9304-412c-88db-28aa99e7c921 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:54.388042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1189/2400] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:00:54 2022] POST /identity/v3/projects => generated 357 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:00:54.392585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335450-1346-4049-af3a-cadf6a207b43 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:54.403493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335450-1346-4049-af3a-cadf6a207b43 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:54.465417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55a62-a894-4643-9dac-c1e1c9ca7392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:54.466090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55a62-a894-4643-9dac-c1e1c9ca7392 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:54.466376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55a62-a894-4643-9dac-c1e1c9ca7392 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:54.466670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec55a62-a894-4643-9dac-c1e1c9ca7392 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:54.467380 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55a62-a894-4643-9dac-c1e1c9ca7392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:54.469922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec55a62-a894-4643-9dac-c1e1c9ca7392 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:54.620868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1212/2401] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:00:54 2022] POST /identity/v3/users => generated 285 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:00:54.625341 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78663a3f-9304-412c-88db-28aa99e7c921 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:54.635578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78663a3f-9304-412c-88db-28aa99e7c921 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:54.640771 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a15f62-e376-457c-8455-2ca145c9ca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:54.641357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a15f62-e376-457c-8455-2ca145c9ca29 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:54.641640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a15f62-e376-457c-8455-2ca145c9ca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:54.641916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a15f62-e376-457c-8455-2ca145c9ca29 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:54.642622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a15f62-e376-457c-8455-2ca145c9ca29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:54.644256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a15f62-e376-457c-8455-2ca145c9ca29 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:54.654710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1190/2402] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:00:54.658857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55a62-a894-4643-9dac-c1e1c9ca7392 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:54.669126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec55a62-a894-4643-9dac-c1e1c9ca7392 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:54.673741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8335239-81c7-4f4b-981a-52cb7a74e679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:54.674806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335239-81c7-4f4b-981a-52cb7a74e679 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:54.675001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335239-81c7-4f4b-981a-52cb7a74e679 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:54.675249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8335239-81c7-4f4b-981a-52cb7a74e679 admin admin] PATH_INFO: `/v3/projects/94e14e61c3f244c990b80786855a6b27/users/8b104d15342a41049f3d66ad22fcdb49/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:54.690884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8335239-81c7-4f4b-981a-52cb7a74e679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e14e61c3f244c990b80786855a6b27, user_id=8b104d15342a41049f3d66ad22fcdb49, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:54.693622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8335239-81c7-4f4b-981a-52cb7a74e679 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:54.793809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1213/2403] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:00:54 2022] PUT /identity/v3/projects/94e14e61c3f244c990b80786855a6b27/users/8b104d15342a41049f3d66ad22fcdb49/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:00:54.798568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a15f62-e376-457c-8455-2ca145c9ca29 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:54.811118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a15f62-e376-457c-8455-2ca145c9ca29 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:54.859379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:54.869584 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fa725-2d29-48d7-8de4-ba3dd3701f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:54.870259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fa725-2d29-48d7-8de4-ba3dd3701f18 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:54.870564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fa725-2d29-48d7-8de4-ba3dd3701f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:54.870826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fa725-2d29-48d7-8de4-ba3dd3701f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:54.871575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fa725-2d29-48d7-8de4-ba3dd3701f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:54.873186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fa725-2d29-48d7-8de4-ba3dd3701f18 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:54.886604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1191/2404] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:00:54.887462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8335239-81c7-4f4b-981a-52cb7a74e679 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:54.899167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8335239-81c7-4f4b-981a-52cb7a74e679 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:54.903792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:54.904680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:54.905783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:54.905783 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] PATH_INFO: `/v3/projects/94e14e61c3f244c990b80786855a6b27/users/8b104d15342a41049f3d66ad22fcdb49/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:54.907837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=94e14e61c3f244c990b80786855a6b27, user_id=8b104d15342a41049f3d66ad22fcdb49, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:54.910537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:54.921151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1214/2405] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:00:54 2022] PUT /identity/v3/projects/94e14e61c3f244c990b80786855a6b27/users/8b104d15342a41049f3d66ad22fcdb49/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:00:54.931315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebde18-67eb-4e15-8793-928694697ba3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:54.931620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebde18-67eb-4e15-8793-928694697ba3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:54.931909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebde18-67eb-4e15-8793-928694697ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:54.959201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-42ebde18-67eb-4e15-8793-928694697ba3 None None] MFA Rules not processed for user `8b104d15342a41049f3d66ad22fcdb49`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:00:54.960451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-42ebde18-67eb-4e15-8793-928694697ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:55.032831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1192/2406] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:00:55.046330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:55.048218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:55.070322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:00:55.107129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb0236-6f69-4112-80ad-dcaf368d86ce admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:55.111084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:55.111778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:55.112114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:55.112344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:55.121719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:55.123161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:55.133140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1215/2407] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:00:55.387787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebde18-67eb-4e15-8793-928694697ba3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:55.398015 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebde18-67eb-4e15-8793-928694697ba3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:55.401749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7a701-7963-4072-bdf1-245d6b107234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:55.402335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a701-7963-4072-bdf1-245d6b107234 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:55.402623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a701-7963-4072-bdf1-245d6b107234 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:55.402905 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a701-7963-4072-bdf1-245d6b107234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:55.461435 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7a701-7963-4072-bdf1-245d6b107234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:55.465227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7a701-7963-4072-bdf1-245d6b107234 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:55.472509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1193/2408] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:00:59.572798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:00:59.574920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:59.647598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd840bd-ecb5-4edd-bd72-7080b27be1e5 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:00:59.653111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:00:59.653762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:00:59.654047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:00:59.654329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:00:59.656750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:00:59.718066 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:00:59.719728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:00:59.728725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1216/2409] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:00:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:01:05.467768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=68 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:09.724497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=73 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:15.376719 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312124a-4377-4cbe-9cba-663b83078378 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:15.377421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312124a-4377-4cbe-9cba-663b83078378 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:15.377678 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312124a-4377-4cbe-9cba-663b83078378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:15.409576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e312124a-4377-4cbe-9cba-663b83078378 None None] MFA Rules not processed for user `1157b91a306540dd98a072b1ea4941b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:15.410732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e312124a-4377-4cbe-9cba-663b83078378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:15.435830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1194/2410] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:16.596016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:16.710264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8479b-c2ed-44df-9c5f-a4c07c64b35d service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:16.715232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:16.716126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:16.716413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:16.716691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] PATH_INFO: `/v3/users/09b367e11986444c83f600c56b511233` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:16.717950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b367e11986444c83f600c56b511233)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:16.720414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:16.947429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b367e11986444c83f600c56b511233'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:16.959654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b367e11986444c83f600c56b511233'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:16.960392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b367e11986444c83f600c56b511233'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:16.961205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b367e11986444c83f600c56b511233'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:16.968658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b367e11986444c83f600c56b511233'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:17.101872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1217/2411] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:01:16 2022] DELETE /identity/v3/users/09b367e11986444c83f600c56b511233 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:17.106346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e140-b680-450e-9b3c-95c7d11c50c5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:17.106801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e140-b680-450e-9b3c-95c7d11c50c5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:17.106918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93e140-b680-450e-9b3c-95c7d11c50c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:17.145589 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2b93e140-b680-450e-9b3c-95c7d11c50c5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:17.146804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2b93e140-b680-450e-9b3c-95c7d11c50c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:17.204475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1195/2412] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:01:17.214572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:17.215954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:17.258799 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:17.292112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edd7fa-40cc-4f1b-b1b9-a140e7a2e215 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:17.296401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47935ec-b1b8-4ab0-af16-5fffd48fdf6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:17.297129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47935ec-b1b8-4ab0-af16-5fffd48fdf6c service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:17.297510 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47935ec-b1b8-4ab0-af16-5fffd48fdf6c service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:17.297830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47935ec-b1b8-4ab0-af16-5fffd48fdf6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:17.308026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47935ec-b1b8-4ab0-af16-5fffd48fdf6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:17.309550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47935ec-b1b8-4ab0-af16-5fffd48fdf6c service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:17.319500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1218/2413] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:01:17.798249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e140-b680-450e-9b3c-95c7d11c50c5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:17.800270 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2b93e140-b680-450e-9b3c-95c7d11c50c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:17.815557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93e140-b680-450e-9b3c-95c7d11c50c5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:17.822153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:17.824683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:17.824683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:17.824683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] PATH_INFO: `/v3/projects/cc06bcf04d47469fb9a210cb2fbd54e5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:17.825924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc06bcf04d47469fb9a210cb2fbd54e5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:17.828458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:17.945866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc06bcf04d47469fb9a210cb2fbd54e5', 'request_id': 'req-7d641e01-a0fc-4652-95c1-ac0d4877b407'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:17.952329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc06bcf04d47469fb9a210cb2fbd54e5', 'request_id': 'req-7d641e01-a0fc-4652-95c1-ac0d4877b407'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:17.958037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc06bcf04d47469fb9a210cb2fbd54e5', 'request_id': 'req-7d641e01-a0fc-4652-95c1-ac0d4877b407'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:18.036644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] The token cache is being invalidate because project cc06bcf04d47469fb9a210cb2fbd54e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:01:18.036936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:18.037715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:18.039391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1196/2414] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:01:17 2022] DELETE /identity/v3/projects/cc06bcf04d47469fb9a210cb2fbd54e5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:18.043404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47935ec-b1b8-4ab0-af16-5fffd48fdf6c service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:18.053846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47935ec-b1b8-4ab0-af16-5fffd48fdf6c service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:18.110036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:18.110905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:18.111200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:18.111509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] PATH_INFO: `/v3/users/8b104d15342a41049f3d66ad22fcdb49` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:18.112697 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b104d15342a41049f3d66ad22fcdb49)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:18.114640 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:18.400161 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b104d15342a41049f3d66ad22fcdb49'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:18.420530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b104d15342a41049f3d66ad22fcdb49'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:18.422034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b104d15342a41049f3d66ad22fcdb49'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:18.422580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b104d15342a41049f3d66ad22fcdb49'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:18.429791 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f1cb488-c9ec-49d8-9504-af8e5bae51d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b104d15342a41049f3d66ad22fcdb49'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:18.498563 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1219/2415] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:01:18 2022] DELETE /identity/v3/users/8b104d15342a41049f3d66ad22fcdb49 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:19.123315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:19.124885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:19.140902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d641e01-a0fc-4652-95c1-ac0d4877b407 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:19.194382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:19.223403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:19.224474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:19.224768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:19.225048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] PATH_INFO: `/v3/projects/94e14e61c3f244c990b80786855a6b27` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:19.226288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94e14e61c3f244c990b80786855a6b27)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:19.228324 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:19.341053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94e14e61c3f244c990b80786855a6b27', 'request_id': 'req-78c15962-819f-4263-82da-4841e64a9278'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:19.348961 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94e14e61c3f244c990b80786855a6b27', 'request_id': 'req-78c15962-819f-4263-82da-4841e64a9278'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:19.354760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94e14e61c3f244c990b80786855a6b27', 'request_id': 'req-78c15962-819f-4263-82da-4841e64a9278'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:19.432073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] The token cache is being invalidate because project 94e14e61c3f244c990b80786855a6b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:01:19.439134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:19.439134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78c15962-819f-4263-82da-4841e64a9278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:19.439134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1197/2416] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:01:19 2022] DELETE /identity/v3/projects/94e14e61c3f244c990b80786855a6b27 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:19.441575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00394f4d-5721-44c6-babb-00791ab362a5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:19.442002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00394f4d-5721-44c6-babb-00791ab362a5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:19.442455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00394f4d-5721-44c6-babb-00791ab362a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:19.511727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-00394f4d-5721-44c6-babb-00791ab362a5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:19.512984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-00394f4d-5721-44c6-babb-00791ab362a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:19.532064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:19.587069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1220/2417] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:19.595211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2de9d-9a97-48a4-8ec7-5ca1bf88321e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:19.595682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2de9d-9a97-48a4-8ec7-5ca1bf88321e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:19.596077 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2de9d-9a97-48a4-8ec7-5ca1bf88321e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:19.616758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c4f2de9d-9a97-48a4-8ec7-5ca1bf88321e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:19.618225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c4f2de9d-9a97-48a4-8ec7-5ca1bf88321e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:19.636931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1198/2418] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:19.641733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00394f4d-5721-44c6-babb-00791ab362a5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:19.651788 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00394f4d-5721-44c6-babb-00791ab362a5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:19.657251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:19.657918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:19.658248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:19.658530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:19.659259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:01:19.659638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:19.661254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:19.674594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1221/2419] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:01:19.678893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2de9d-9a97-48a4-8ec7-5ca1bf88321e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:19.688412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2de9d-9a97-48a4-8ec7-5ca1bf88321e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:19.693297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06805-5aa4-40e1-9b3b-dd1fc1dc337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:19.693900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06805-5aa4-40e1-9b3b-dd1fc1dc337b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:19.694188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06805-5aa4-40e1-9b3b-dd1fc1dc337b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:19.694494 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06805-5aa4-40e1-9b3b-dd1fc1dc337b admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:19.695189 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec06805-5aa4-40e1-9b3b-dd1fc1dc337b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:19.697155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec06805-5aa4-40e1-9b3b-dd1fc1dc337b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:19.804279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1199/2420] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:01:19 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:01:19.808659 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:19.819384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256fd33-7b84-40c2-a3b3-b1e85c2d05bf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:19.895490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee61692-e973-49f1-94ba-3f98bc7711f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:19.896118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee61692-e973-49f1-94ba-3f98bc7711f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:19.896440 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee61692-e973-49f1-94ba-3f98bc7711f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:19.896743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee61692-e973-49f1-94ba-3f98bc7711f1 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:19.897429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee61692-e973-49f1-94ba-3f98bc7711f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:19.899706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee61692-e973-49f1-94ba-3f98bc7711f1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.008020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1222/2421] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:01:19 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:01:20.012294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06805-5aa4-40e1-9b3b-dd1fc1dc337b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.023781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06805-5aa4-40e1-9b3b-dd1fc1dc337b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.028408 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfdb79-d4a2-4466-bbf4-cef86a4e2a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.029071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfdb79-d4a2-4466-bbf4-cef86a4e2a34 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.029495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfdb79-d4a2-4466-bbf4-cef86a4e2a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.029740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfdb79-d4a2-4466-bbf4-cef86a4e2a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.031003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfdb79-d4a2-4466-bbf4-cef86a4e2a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:20.032672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfdb79-d4a2-4466-bbf4-cef86a4e2a34 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.043734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1200/2422] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:01:20.049025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee61692-e973-49f1-94ba-3f98bc7711f1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.059733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee61692-e973-49f1-94ba-3f98bc7711f1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.064465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf238bc-5ed5-48f5-8219-64b18b2b9ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.065133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf238bc-5ed5-48f5-8219-64b18b2b9ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.065421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf238bc-5ed5-48f5-8219-64b18b2b9ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.065700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf238bc-5ed5-48f5-8219-64b18b2b9ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.066420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf238bc-5ed5-48f5-8219-64b18b2b9ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:20.068090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf238bc-5ed5-48f5-8219-64b18b2b9ab5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.079000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1223/2423] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:01:20.083235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfdb79-d4a2-4466-bbf4-cef86a4e2a34 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.095157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfdb79-d4a2-4466-bbf4-cef86a4e2a34 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.100181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad0f9d-537c-4577-bc57-fa500bc54f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.101145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0f9d-537c-4577-bc57-fa500bc54f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.101495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0f9d-537c-4577-bc57-fa500bc54f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.101830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0f9d-537c-4577-bc57-fa500bc54f63 admin admin] PATH_INFO: `/v3/projects/63d05cf98f74419f82e7086108319870/users/51951d7a5ae24b01825d042d5b1f8308/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.118732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad0f9d-537c-4577-bc57-fa500bc54f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d05cf98f74419f82e7086108319870, user_id=51951d7a5ae24b01825d042d5b1f8308, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:20.121567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad0f9d-537c-4577-bc57-fa500bc54f63 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.191977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1201/2424] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:01:20 2022] PUT /identity/v3/projects/63d05cf98f74419f82e7086108319870/users/51951d7a5ae24b01825d042d5b1f8308/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:20.203126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb3988-70b4-4177-a61f-ce8bb6ac3bab None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.203458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb3988-70b4-4177-a61f-ce8bb6ac3bab None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.203810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb3988-70b4-4177-a61f-ce8bb6ac3bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.230808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3adb3988-70b4-4177-a61f-ce8bb6ac3bab None None] MFA Rules not processed for user `51951d7a5ae24b01825d042d5b1f8308`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:20.232102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3adb3988-70b4-4177-a61f-ce8bb6ac3bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:20.297283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1224/2425] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:20.301929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad0f9d-537c-4577-bc57-fa500bc54f63 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.377220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad0f9d-537c-4577-bc57-fa500bc54f63 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.382409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41034be-c291-49fd-9245-5c37bbcc5dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.383561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41034be-c291-49fd-9245-5c37bbcc5dbc admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.383561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41034be-c291-49fd-9245-5c37bbcc5dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.390801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41034be-c291-49fd-9245-5c37bbcc5dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.390801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41034be-c291-49fd-9245-5c37bbcc5dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:20.390801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41034be-c291-49fd-9245-5c37bbcc5dbc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.392655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:20.518201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1202/2426] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:01:20 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:01:20.518329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb3988-70b4-4177-a61f-ce8bb6ac3bab None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.527433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb3988-70b4-4177-a61f-ce8bb6ac3bab None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.539205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:20.583479 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf53e6-a4ff-4e0f-bfc0-6cbc47b67d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.584148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf53e6-a4ff-4e0f-bfc0-6cbc47b67d13 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.584431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf53e6-a4ff-4e0f-bfc0-6cbc47b67d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.584703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf53e6-a4ff-4e0f-bfc0-6cbc47b67d13 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.585394 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf53e6-a4ff-4e0f-bfc0-6cbc47b67d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:20.587702 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf53e6-a4ff-4e0f-bfc0-6cbc47b67d13 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.697436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1225/2427] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:01:20 2022] POST /identity/v3/users => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:01:20.705639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41034be-c291-49fd-9245-5c37bbcc5dbc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.731342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41034be-c291-49fd-9245-5c37bbcc5dbc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.744491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b31503-11c0-49ae-98c8-64d16deaff9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.745193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b31503-11c0-49ae-98c8-64d16deaff9b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.746275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b31503-11c0-49ae-98c8-64d16deaff9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.746623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b31503-11c0-49ae-98c8-64d16deaff9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.747715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b31503-11c0-49ae-98c8-64d16deaff9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:20.749739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b31503-11c0-49ae-98c8-64d16deaff9b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.768287 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf53e6-a4ff-4e0f-bfc0-6cbc47b67d13 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.768721 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1203/2428] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:01:20.779105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf53e6-a4ff-4e0f-bfc0-6cbc47b67d13 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.786397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b033-4628-4861-84a4-95cb6cf4e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.787418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b033-4628-4861-84a4-95cb6cf4e869 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.787725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b033-4628-4861-84a4-95cb6cf4e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.789563 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b033-4628-4861-84a4-95cb6cf4e869 admin admin] PATH_INFO: `/v3/projects/fe0f0bd21b7147918e500b5f043fb5d6/users/c46e57aac2d946bea64c9a53d483fd67/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.816472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305b033-4628-4861-84a4-95cb6cf4e869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe0f0bd21b7147918e500b5f043fb5d6, user_id=c46e57aac2d946bea64c9a53d483fd67, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:20.819438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305b033-4628-4861-84a4-95cb6cf4e869 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.882932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1226/2429] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:01:20 2022] PUT /identity/v3/projects/fe0f0bd21b7147918e500b5f043fb5d6/users/c46e57aac2d946bea64c9a53d483fd67/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:20.887126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b31503-11c0-49ae-98c8-64d16deaff9b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.898856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b31503-11c0-49ae-98c8-64d16deaff9b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.961091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035706a5-4543-4fb3-83dd-3919f107226c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.961766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035706a5-4543-4fb3-83dd-3919f107226c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.962058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035706a5-4543-4fb3-83dd-3919f107226c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.962342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035706a5-4543-4fb3-83dd-3919f107226c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.963093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035706a5-4543-4fb3-83dd-3919f107226c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:20.964787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035706a5-4543-4fb3-83dd-3919f107226c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:20.978607 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1204/2430] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:01:20.979911 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b033-4628-4861-84a4-95cb6cf4e869 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:20.990489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b033-4628-4861-84a4-95cb6cf4e869 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:20.995239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:20.996155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:20.996443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:20.996720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] PATH_INFO: `/v3/projects/fe0f0bd21b7147918e500b5f043fb5d6/users/c46e57aac2d946bea64c9a53d483fd67/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:20.998718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe0f0bd21b7147918e500b5f043fb5d6, user_id=c46e57aac2d946bea64c9a53d483fd67, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:21.001454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:21.012250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1227/2431] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:01:20 2022] PUT /identity/v3/projects/fe0f0bd21b7147918e500b5f043fb5d6/users/c46e57aac2d946bea64c9a53d483fd67/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:21.024041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5be70-cd7f-46ce-a906-9fc7e479a393 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:21.024338 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5be70-cd7f-46ce-a906-9fc7e479a393 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:21.024624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5be70-cd7f-46ce-a906-9fc7e479a393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:21.052349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-02b5be70-cd7f-46ce-a906-9fc7e479a393 None None] MFA Rules not processed for user `c46e57aac2d946bea64c9a53d483fd67`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:21.053573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-02b5be70-cd7f-46ce-a906-9fc7e479a393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:21.124112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1205/2432] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:21.134768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:21.136558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:21.204314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6f9d6-3245-4765-887a-0746b8381690 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:21.208124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7182426-8e48-47ae-8ad5-5c40d5fc2ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:21.209545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7182426-8e48-47ae-8ad5-5c40d5fc2ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:21.209545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7182426-8e48-47ae-8ad5-5c40d5fc2ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:21.209545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7182426-8e48-47ae-8ad5-5c40d5fc2ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:21.220010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7182426-8e48-47ae-8ad5-5c40d5fc2ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:21.221500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7182426-8e48-47ae-8ad5-5c40d5fc2ea4 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:21.231031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1228/2433] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:01:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:01:21.334587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5be70-cd7f-46ce-a906-9fc7e479a393 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:21.345475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5be70-cd7f-46ce-a906-9fc7e479a393 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:21.349175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd6e0a-bdbb-4c7c-8a88-f16710193913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:21.349865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6e0a-bdbb-4c7c-8a88-f16710193913 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:21.350149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6e0a-bdbb-4c7c-8a88-f16710193913 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:21.350425 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6e0a-bdbb-4c7c-8a88-f16710193913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:21.406720 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:21.410934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd6e0a-bdbb-4c7c-8a88-f16710193913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:21.412444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd6e0a-bdbb-4c7c-8a88-f16710193913 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:21.422233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1206/2434] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:01:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:01:25.570658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7182426-8e48-47ae-8ad5-5c40d5fc2ea4 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:25.633403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7182426-8e48-47ae-8ad5-5c40d5fc2ea4 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:25.638017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:25.638887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:25.639175 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:25.639492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] PATH_INFO: `/v3/users/51951d7a5ae24b01825d042d5b1f8308` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:25.641012 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51951d7a5ae24b01825d042d5b1f8308)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:25.642955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:25.655474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:25.821359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51951d7a5ae24b01825d042d5b1f8308'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:25.833124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51951d7a5ae24b01825d042d5b1f8308'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:25.844386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51951d7a5ae24b01825d042d5b1f8308'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:25.844386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51951d7a5ae24b01825d042d5b1f8308'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:25.844386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51951d7a5ae24b01825d042d5b1f8308'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:25.899661 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1229/2435] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:01:25 2022] DELETE /identity/v3/users/51951d7a5ae24b01825d042d5b1f8308 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:25.909914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715bf9-0b14-41b6-8cd4-01ba25f7f3b5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:25.910415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715bf9-0b14-41b6-8cd4-01ba25f7f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:25.910522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715bf9-0b14-41b6-8cd4-01ba25f7f3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:25.928043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7e715bf9-0b14-41b6-8cd4-01ba25f7f3b5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:25.929246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7e715bf9-0b14-41b6-8cd4-01ba25f7f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:25.991795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1207/2436] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:26.001551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:26.003065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:26.061523 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a783f7-a011-4ac0-9106-6545ecad195f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:26.065203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36514c-0e0a-41ef-bc6b-7e7a427e9904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:26.065855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36514c-0e0a-41ef-bc6b-7e7a427e9904 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:26.066194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36514c-0e0a-41ef-bc6b-7e7a427e9904 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:26.066477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36514c-0e0a-41ef-bc6b-7e7a427e9904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:26.075554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36514c-0e0a-41ef-bc6b-7e7a427e9904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:26.076912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b36514c-0e0a-41ef-bc6b-7e7a427e9904 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:26.087163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1230/2437] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:01:26.548545 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e715bf9-0b14-41b6-8cd4-01ba25f7f3b5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:26.548789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7e715bf9-0b14-41b6-8cd4-01ba25f7f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:26.562074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e715bf9-0b14-41b6-8cd4-01ba25f7f3b5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:26.569829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:26.570706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:26.571037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:26.571350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] PATH_INFO: `/v3/projects/63d05cf98f74419f82e7086108319870` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:26.572665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d05cf98f74419f82e7086108319870)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:26.575346 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:26.718863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d05cf98f74419f82e7086108319870', 'request_id': 'req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:26.725712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d05cf98f74419f82e7086108319870', 'request_id': 'req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:26.731453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d05cf98f74419f82e7086108319870', 'request_id': 'req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:26.814910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] The token cache is being invalidate because project 63d05cf98f74419f82e7086108319870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:01:26.815201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:26.816004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:26.817642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1208/2438] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:01:26 2022] DELETE /identity/v3/projects/63d05cf98f74419f82e7086108319870 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:26.822022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36514c-0e0a-41ef-bc6b-7e7a427e9904 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:26.832215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36514c-0e0a-41ef-bc6b-7e7a427e9904 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:26.890803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:26.891722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:26.892023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:26.892300 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] PATH_INFO: `/v3/users/c46e57aac2d946bea64c9a53d483fd67` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:26.893525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46e57aac2d946bea64c9a53d483fd67)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:26.895503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:27.076319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:27.085452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46e57aac2d946bea64c9a53d483fd67'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.097260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46e57aac2d946bea64c9a53d483fd67'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.097957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46e57aac2d946bea64c9a53d483fd67'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.098698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46e57aac2d946bea64c9a53d483fd67'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.105670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6c0c8dd4-bd24-4813-ada0-214e6aeebd16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46e57aac2d946bea64c9a53d483fd67'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.214322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1231/2439] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:01:26 2022] DELETE /identity/v3/users/c46e57aac2d946bea64c9a53d483fd67 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:27.609468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:27.610866 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:27.624067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf3b0f-eb2c-4c0b-a5d0-a3349ad08b69 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:27.683389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:27.684254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:27.684542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:27.684828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] PATH_INFO: `/v3/projects/fe0f0bd21b7147918e500b5f043fb5d6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:27.686184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe0f0bd21b7147918e500b5f043fb5d6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:27.688206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:27.710698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:27.821051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe0f0bd21b7147918e500b5f043fb5d6', 'request_id': 'req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.827414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe0f0bd21b7147918e500b5f043fb5d6', 'request_id': 'req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.833813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe0f0bd21b7147918e500b5f043fb5d6', 'request_id': 'req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.904836 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] The token cache is being invalidate because project fe0f0bd21b7147918e500b5f043fb5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:01:27.905060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.905675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3ab9aca7-636a-4f19-bbbe-4793e8231c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:27.907817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1209/2440] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:01:27 2022] DELETE /identity/v3/projects/fe0f0bd21b7147918e500b5f043fb5d6 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:27.912730 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35228fbd-c380-40c0-9e27-2d751903e93c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:27.913202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35228fbd-c380-40c0-9e27-2d751903e93c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:27.913306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35228fbd-c380-40c0-9e27-2d751903e93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:27.930576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-35228fbd-c380-40c0-9e27-2d751903e93c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:27.931831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-35228fbd-c380-40c0-9e27-2d751903e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:27.991139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1232/2441] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:27.998896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870f3f-4c33-4cf4-92cb-8a0eb0ec938b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:27.999197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870f3f-4c33-4cf4-92cb-8a0eb0ec938b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:27.999507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870f3f-4c33-4cf4-92cb-8a0eb0ec938b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.018500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d2870f3f-4c33-4cf4-92cb-8a0eb0ec938b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:28.019729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d2870f3f-4c33-4cf4-92cb-8a0eb0ec938b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:28.029071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1210/2442] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:28.033467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35228fbd-c380-40c0-9e27-2d751903e93c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:28.043114 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35228fbd-c380-40c0-9e27-2d751903e93c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:28.047860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:28.048543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:28.048834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:28.049117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.049835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:01:28.050153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:28.051796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:28.063873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1233/2443] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:01:28.067998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870f3f-4c33-4cf4-92cb-8a0eb0ec938b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:28.077246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870f3f-4c33-4cf4-92cb-8a0eb0ec938b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:28.082624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb31f72-9c27-4cb8-a8d4-fab83f5d5c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:28.083269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb31f72-9c27-4cb8-a8d4-fab83f5d5c81 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:28.083590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb31f72-9c27-4cb8-a8d4-fab83f5d5c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:28.083898 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb31f72-9c27-4cb8-a8d4-fab83f5d5c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.084583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb31f72-9c27-4cb8-a8d4-fab83f5d5c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:28.086773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb31f72-9c27-4cb8-a8d4-fab83f5d5c81 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:28.171682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1211/2444] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:01:28 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:01:28.177120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:28.188083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe18e6-f333-40dd-9f74-0d5fb3b43520 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:28.245409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb990a33-bb4e-4e55-96a9-923dce2c9867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:28.246130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990a33-bb4e-4e55-96a9-923dce2c9867 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:28.246459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990a33-bb4e-4e55-96a9-923dce2c9867 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:28.246679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb990a33-bb4e-4e55-96a9-923dce2c9867 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.247437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb990a33-bb4e-4e55-96a9-923dce2c9867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:28.249381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb990a33-bb4e-4e55-96a9-923dce2c9867 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:28.378935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1234/2445] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:01:28 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:01:28.385772 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb31f72-9c27-4cb8-a8d4-fab83f5d5c81 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:28.404232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb31f72-9c27-4cb8-a8d4-fab83f5d5c81 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:28.409410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bffc8-87bf-49ec-9167-e72e1d12e0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:28.410092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bffc8-87bf-49ec-9167-e72e1d12e0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:28.410378 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bffc8-87bf-49ec-9167-e72e1d12e0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:28.410660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bffc8-87bf-49ec-9167-e72e1d12e0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.411407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bffc8-87bf-49ec-9167-e72e1d12e0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:28.413105 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bffc8-87bf-49ec-9167-e72e1d12e0e4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:28.428911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1212/2446] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:01:28.431692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb990a33-bb4e-4e55-96a9-923dce2c9867 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:28.442714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb990a33-bb4e-4e55-96a9-923dce2c9867 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:28.447714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c2d0b-d1bc-46b7-b2d3-08db239eda4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:28.448455 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2d0b-d1bc-46b7-b2d3-08db239eda4d admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:28.448846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2d0b-d1bc-46b7-b2d3-08db239eda4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:28.449018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2d0b-d1bc-46b7-b2d3-08db239eda4d admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.449773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c2d0b-d1bc-46b7-b2d3-08db239eda4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:28.451408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c2d0b-d1bc-46b7-b2d3-08db239eda4d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:28.463017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1235/2447] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:01:28.466937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bffc8-87bf-49ec-9167-e72e1d12e0e4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:28.477318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bffc8-87bf-49ec-9167-e72e1d12e0e4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:28.482320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:28.483207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:28.483537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:28.483820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] PATH_INFO: `/v3/projects/fb47f9147783478f913d81bff5a4a633/users/2f7a278ff970428c85a9039d5c0d4777/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.505022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb47f9147783478f913d81bff5a4a633, user_id=2f7a278ff970428c85a9039d5c0d4777, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:28.507806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:28.603280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1213/2448] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:01:28 2022] PUT /identity/v3/projects/fb47f9147783478f913d81bff5a4a633/users/2f7a278ff970428c85a9039d5c0d4777/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:01:28.614235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d357360-3e82-47b6-9918-7e3bb5057091 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:28.614527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d357360-3e82-47b6-9918-7e3bb5057091 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:28.614810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d357360-3e82-47b6-9918-7e3bb5057091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.643874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6d357360-3e82-47b6-9918-7e3bb5057091 None None] MFA Rules not processed for user `2f7a278ff970428c85a9039d5c0d4777`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:28.645101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6d357360-3e82-47b6-9918-7e3bb5057091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:28.650255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:28.709486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1236/2449] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:28.721137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:28.722826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:28.779929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c557469-0ec2-4aad-8df2-b36d1ea3adec admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:28.784281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f18d0f5f634d84a99fc3e46963a8b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:28.785033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:28.785349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:28.785629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:28.794593 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:28.796072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:28.807085 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1214/2450] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:01:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:01:38.802546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:39.025465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=24 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:44.639015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0f261-daa6-4af5-b585-afe4b7ccfa28 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:44.639647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0f261-daa6-4af5-b585-afe4b7ccfa28 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:44.639647 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0f261-daa6-4af5-b585-afe4b7ccfa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:44.669411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-29c0f261-daa6-4af5-b585-afe4b7ccfa28 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:44.670648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-29c0f261-daa6-4af5-b585-afe4b7ccfa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:44.733967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1237/2451] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 16:01:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:44.909130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:44.910550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:44.979101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad45ca-3d70-4e57-988a-fc77d5fbc66f service glance] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:44.984625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad504c5-4eee-4bb6-94f9-ab906fb2cbdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:44.985213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad504c5-4eee-4bb6-94f9-ab906fb2cbdf service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:44.985496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad504c5-4eee-4bb6-94f9-ab906fb2cbdf service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:44.985770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad504c5-4eee-4bb6-94f9-ab906fb2cbdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:44.994109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad504c5-4eee-4bb6-94f9-ab906fb2cbdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:44.995745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad504c5-4eee-4bb6-94f9-ab906fb2cbdf service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:45.009090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1215/2452] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 16:01:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 16:01:54.727311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=32 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:55.001755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=32 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:57.721714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0f261-daa6-4af5-b585-afe4b7ccfa28 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:57.807051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0f261-daa6-4af5-b585-afe4b7ccfa28 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:57.812275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:57.814053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:57.814513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:57.815087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] PATH_INFO: `/v3/users/2f7a278ff970428c85a9039d5c0d4777` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:57.816769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7a278ff970428c85a9039d5c0d4777)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:57.819153 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:58.409742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7a278ff970428c85a9039d5c0d4777'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:58.421481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7a278ff970428c85a9039d5c0d4777'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:58.422235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7a278ff970428c85a9039d5c0d4777'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:58.422860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7a278ff970428c85a9039d5c0d4777'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:58.429881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7a278ff970428c85a9039d5c0d4777'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:58.545084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1238/2453] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:01:57 2022] DELETE /identity/v3/users/2f7a278ff970428c85a9039d5c0d4777 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:58.549602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c50457-ff25-42bb-9496-dde689cc85c0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:58.550066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c50457-ff25-42bb-9496-dde689cc85c0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:58.550259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c50457-ff25-42bb-9496-dde689cc85c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:58.570253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-46c50457-ff25-42bb-9496-dde689cc85c0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:58.573596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-46c50457-ff25-42bb-9496-dde689cc85c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:58.630049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1216/2454] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:58.639719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:58.641609 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:58.721170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139117a3-2447-4db3-b25a-b47623ba1f0a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:58.724928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23a7e2-5b25-4aca-acf4-59b3bdfa22a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:58.725579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23a7e2-5b25-4aca-acf4-59b3bdfa22a5 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:58.725866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23a7e2-5b25-4aca-acf4-59b3bdfa22a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:58.726146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23a7e2-5b25-4aca-acf4-59b3bdfa22a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:58.735802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23a7e2-5b25-4aca-acf4-59b3bdfa22a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:58.736984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23a7e2-5b25-4aca-acf4-59b3bdfa22a5 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:58.747513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1239/2455] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:01:59.219609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c50457-ff25-42bb-9496-dde689cc85c0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:59.221079 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-46c50457-ff25-42bb-9496-dde689cc85c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:59.251082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c50457-ff25-42bb-9496-dde689cc85c0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:59.261233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:59.262655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:59.263052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:59.263216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] PATH_INFO: `/v3/projects/fb47f9147783478f913d81bff5a4a633` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:59.264473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb47f9147783478f913d81bff5a4a633)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:59.266484 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:59.500935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb47f9147783478f913d81bff5a4a633', 'request_id': 'req-a7d09dee-2237-4873-b171-ab8564a78673'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:59.500935 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb47f9147783478f913d81bff5a4a633', 'request_id': 'req-a7d09dee-2237-4873-b171-ab8564a78673'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:59.509487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb47f9147783478f913d81bff5a4a633', 'request_id': 'req-a7d09dee-2237-4873-b171-ab8564a78673'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:59.602052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] The token cache is being invalidate because project fb47f9147783478f913d81bff5a4a633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:01:59.602542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:59.603048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a7d09dee-2237-4873-b171-ab8564a78673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:01:59.604795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1217/2456] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:01:59 2022] DELETE /identity/v3/projects/fb47f9147783478f913d81bff5a4a633 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:01:59.610330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de91d-dd61-48d6-819a-3e8b53dae060 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:59.610790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de91d-dd61-48d6-819a-3e8b53dae060 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:59.610891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de91d-dd61-48d6-819a-3e8b53dae060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:59.617629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:01:59.630511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cc4de91d-dd61-48d6-819a-3e8b53dae060 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:59.631760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cc4de91d-dd61-48d6-819a-3e8b53dae060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:59.689938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1240/2457] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:59.701700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37328662-9b3c-4e6c-87f8-641c01309dc6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:59.701700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37328662-9b3c-4e6c-87f8-641c01309dc6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:59.701700 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37328662-9b3c-4e6c-87f8-641c01309dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:59.718528 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-37328662-9b3c-4e6c-87f8-641c01309dc6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:01:59.719803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-37328662-9b3c-4e6c-87f8-641c01309dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:01:59.728056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1218/2458] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:01:59.732163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4de91d-dd61-48d6-819a-3e8b53dae060 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:59.742705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4de91d-dd61-48d6-819a-3e8b53dae060 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:59.747285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:59.747969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:59.748259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:59.748544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:59.749294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:01:59.749610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:59.751310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:59.762632 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1241/2459] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:01:59.767222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37328662-9b3c-4e6c-87f8-641c01309dc6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:59.777041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37328662-9b3c-4e6c-87f8-641c01309dc6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:59.782197 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136d829-3c22-4b9d-b217-058b7e78d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:59.782838 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136d829-3c22-4b9d-b217-058b7e78d736 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:59.783123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136d829-3c22-4b9d-b217-058b7e78d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:59.783609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136d829-3c22-4b9d-b217-058b7e78d736 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:59.784291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136d829-3c22-4b9d-b217-058b7e78d736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:59.786805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136d829-3c22-4b9d-b217-058b7e78d736 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:01:59.896368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1219/2460] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:01:59 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:01:59.900547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:01:59.911254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d2e3-137a-4d2e-a824-bb2bbed2039c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:01:59.966718 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9f4d6-fe5b-42d9-a1e6-fc995b938a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:01:59.967434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f4d6-fe5b-42d9-a1e6-fc995b938a7c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:01:59.967727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f4d6-fe5b-42d9-a1e6-fc995b938a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:01:59.968586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f4d6-fe5b-42d9-a1e6-fc995b938a7c admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:01:59.969250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9f4d6-fe5b-42d9-a1e6-fc995b938a7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:01:59.971226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9f4d6-fe5b-42d9-a1e6-fc995b938a7c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:00.065274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1242/2461] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:01:59 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:02:00.069404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136d829-3c22-4b9d-b217-058b7e78d736 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:00.080273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136d829-3c22-4b9d-b217-058b7e78d736 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:00.084870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5e575-d7b2-42ae-99ca-ecb6904de4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:00.085520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5e575-d7b2-42ae-99ca-ecb6904de4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:00.085803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5e575-d7b2-42ae-99ca-ecb6904de4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:00.086081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5e575-d7b2-42ae-99ca-ecb6904de4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:00.087052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5e575-d7b2-42ae-99ca-ecb6904de4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:00.088694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5e575-d7b2-42ae-99ca-ecb6904de4c1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:00.099807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1220/2462] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:02:00.103753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9f4d6-fe5b-42d9-a1e6-fc995b938a7c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:00.114105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9f4d6-fe5b-42d9-a1e6-fc995b938a7c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:00.118963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3eacc-08a3-4895-9b1d-5cc43ae47178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:00.123603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3eacc-08a3-4895-9b1d-5cc43ae47178 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:00.123603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3eacc-08a3-4895-9b1d-5cc43ae47178 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:00.123603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3eacc-08a3-4895-9b1d-5cc43ae47178 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:00.123603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3eacc-08a3-4895-9b1d-5cc43ae47178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:00.123603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3eacc-08a3-4895-9b1d-5cc43ae47178 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:00.132805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1243/2463] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:02:00.136855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5e575-d7b2-42ae-99ca-ecb6904de4c1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:00.147312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5e575-d7b2-42ae-99ca-ecb6904de4c1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:00.151848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:00.152712 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:00.152999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:00.153276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] PATH_INFO: `/v3/projects/6857104436b34711b09ed7da81c5840d/users/8f31c861aa9d4d508f676db9eea6d8fa/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:00.168335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6857104436b34711b09ed7da81c5840d, user_id=8f31c861aa9d4d508f676db9eea6d8fa, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:00.171042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:00.241517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1221/2464] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:02:00 2022] PUT /identity/v3/projects/6857104436b34711b09ed7da81c5840d/users/8f31c861aa9d4d508f676db9eea6d8fa/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:02:00.252414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ab6d-f00f-47ed-a6b8-8777c01b4c1c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:00.252714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ab6d-f00f-47ed-a6b8-8777c01b4c1c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:00.253023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11ab6d-f00f-47ed-a6b8-8777c01b4c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:00.279854 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bd11ab6d-f00f-47ed-a6b8-8777c01b4c1c None None] MFA Rules not processed for user `8f31c861aa9d4d508f676db9eea6d8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:00.281086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bd11ab6d-f00f-47ed-a6b8-8777c01b4c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:00.340009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:00.345064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1244/2465] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:00.355815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:00.357481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:00.383804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:00.437175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce8989-39ec-4aaf-8711-2a02c5ee6420 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:00.441178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77633f88-8eb2-4b08-995d-d6282561725f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:00.441830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633f88-8eb2-4b08-995d-d6282561725f service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:00.442119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633f88-8eb2-4b08-995d-d6282561725f service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:00.442404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633f88-8eb2-4b08-995d-d6282561725f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:00.451261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633f88-8eb2-4b08-995d-d6282561725f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:00.453878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633f88-8eb2-4b08-995d-d6282561725f service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:00.468017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1222/2466] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:02:03.213398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11ab6d-f00f-47ed-a6b8-8777c01b4c1c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:03.216101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bd11ab6d-f00f-47ed-a6b8-8777c01b4c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:03.325400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11ab6d-f00f-47ed-a6b8-8777c01b4c1c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:03.333135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5cc10-2eb4-4679-a1da-8dd8b596304a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:03.333795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5cc10-2eb4-4679-a1da-8dd8b596304a service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:03.334089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5cc10-2eb4-4679-a1da-8dd8b596304a service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:03.334398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5cc10-2eb4-4679-a1da-8dd8b596304a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:03.336550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-08e5cc10-2eb4-4679-a1da-8dd8b596304a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:03.348565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5cc10-2eb4-4679-a1da-8dd8b596304a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:03.350169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5cc10-2eb4-4679-a1da-8dd8b596304a service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:03.360769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1245/2467] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:02:09.023020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620447c-faa8-455c-82f6-2645056b401a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.023621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620447c-faa8-455c-82f6-2645056b401a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.023621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620447c-faa8-455c-82f6-2645056b401a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.025113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1223/2468] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 16:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 16:02:09.031102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd12bbe-1fa5-47d1-a8b4-aeee9954c2d4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.031616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd12bbe-1fa5-47d1-a8b4-aeee9954c2d4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.031722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd12bbe-1fa5-47d1-a8b4-aeee9954c2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.064125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3fd12bbe-1fa5-47d1-a8b4-aeee9954c2d4 None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:09.065389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3fd12bbe-1fa5-47d1-a8b4-aeee9954c2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:09.147414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1246/2469] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 16:02:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:09.159109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620447c-faa8-455c-82f6-2645056b401a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:09.161469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9620447c-faa8-455c-82f6-2645056b401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:09.220400 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620447c-faa8-455c-82f6-2645056b401a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:09.224497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97fcc1-f254-4f64-9a83-43171587a771 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f18d0f5f634d84a99fc3e46963a8b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:09.225140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97fcc1-f254-4f64-9a83-43171587a771 service glance] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.225428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97fcc1-f254-4f64-9a83-43171587a771 service glance] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.225708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97fcc1-f254-4f64-9a83-43171587a771 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.235847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97fcc1-f254-4f64-9a83-43171587a771 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:09.237323 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97fcc1-f254-4f64-9a83-43171587a771 service glance] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:09.248831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1224/2470] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:02:09.489754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48c0a1-5571-4206-aa5c-5f0ec106cbd6 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.489967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48c0a1-5571-4206-aa5c-5f0ec106cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.490248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48c0a1-5571-4206-aa5c-5f0ec106cbd6 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.491691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1247/2471] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 16:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 16:02:09.496823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c301-8049-476a-9ded-d5f8d4bf1bfe None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.498883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c301-8049-476a-9ded-d5f8d4bf1bfe None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.498883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c301-8049-476a-9ded-d5f8d4bf1bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.515832 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0f59c301-8049-476a-9ded-d5f8d4bf1bfe None None] MFA Rules not processed for user `9d20de9c921942d1afeb63e87272f782`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:09.517031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0f59c301-8049-476a-9ded-d5f8d4bf1bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:09.568809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1225/2472] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 16:02:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:09.580907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48c0a1-5571-4206-aa5c-5f0ec106cbd6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:09.582593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5a48c0a1-5571-4206-aa5c-5f0ec106cbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:09.596603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48c0a1-5571-4206-aa5c-5f0ec106cbd6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:09.600801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076b4b3-ee22-41f2-9940-650b60812eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:09.601913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b4b3-ee22-41f2-9940-650b60812eaf service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.601913 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b4b3-ee22-41f2-9940-650b60812eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.602126 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b4b3-ee22-41f2-9940-650b60812eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.608936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076b4b3-ee22-41f2-9940-650b60812eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:09.610354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076b4b3-ee22-41f2-9940-650b60812eaf service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:09.618096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1248/2473] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:02:09.704352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d44100-6dbb-4205-ae72-39a0df5eda7e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.704644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d44100-6dbb-4205-ae72-39a0df5eda7e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.704947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d44100-6dbb-4205-ae72-39a0df5eda7e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.706498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1226/2474] 10.222.0.98 () {54 vars in 921 bytes} [Fri Jul 15 16:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 16:02:09.712514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e62c8-70a5-43f7-8f4c-b5372e6307ff None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.712805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e62c8-70a5-43f7-8f4c-b5372e6307ff None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.713084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e62c8-70a5-43f7-8f4c-b5372e6307ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.731638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-425e62c8-70a5-43f7-8f4c-b5372e6307ff None None] MFA Rules not processed for user `bf670978cc2e4c529093a191bc26f885`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:09.732752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-425e62c8-70a5-43f7-8f4c-b5372e6307ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:09.742077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1249/2475] 10.222.0.98 () {58 vars in 1018 bytes} [Fri Jul 15 16:02:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:09.752642 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d44100-6dbb-4205-ae72-39a0df5eda7e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:09.754313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-05d44100-6dbb-4205-ae72-39a0df5eda7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:09.767645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d44100-6dbb-4205-ae72-39a0df5eda7e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:09.772090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:09.772734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:09.773023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:09.773303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:09.783768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:09.785221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:09.796835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1227/2476] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:02:12.228496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1db59-3ab6-4ee1-87a6-ab5e7d920576 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:12.228947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1db59-3ab6-4ee1-87a6-ab5e7d920576 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:12.229227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1db59-3ab6-4ee1-87a6-ab5e7d920576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:12.246608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-09f1db59-3ab6-4ee1-87a6-ab5e7d920576 None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:12.247835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-09f1db59-3ab6-4ee1-87a6-ab5e7d920576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:12.260987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:12.302894 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1250/2477] 10.222.0.98 () {58 vars in 1009 bytes} [Fri Jul 15 16:02:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:12.314904 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:12.316352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:12.363261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:12.387098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a04d0f-cf80-4069-95e2-63c74e783004 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:12.392410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a764a-cfdd-4ddc-805b-efcefd14e513 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:12.393061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a764a-cfdd-4ddc-805b-efcefd14e513 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:12.393350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a764a-cfdd-4ddc-805b-efcefd14e513 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:12.393631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a764a-cfdd-4ddc-805b-efcefd14e513 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:12.401941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a764a-cfdd-4ddc-805b-efcefd14e513 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:12.403714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a764a-cfdd-4ddc-805b-efcefd14e513 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:12.412366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1228/2478] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 16:02:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 16:02:12.970965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1db59-3ab6-4ee1-87a6-ab5e7d920576 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:13.042181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1db59-3ab6-4ee1-87a6-ab5e7d920576 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:13.047348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:13.048244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:13.048586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:13.048871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] PATH_INFO: `/v3/users/8f31c861aa9d4d508f676db9eea6d8fa` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:13.050086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f31c861aa9d4d508f676db9eea6d8fa)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:13.052087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:13.343165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f31c861aa9d4d508f676db9eea6d8fa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:13.359407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f31c861aa9d4d508f676db9eea6d8fa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:13.360443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f31c861aa9d4d508f676db9eea6d8fa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:13.361306 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f31c861aa9d4d508f676db9eea6d8fa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:13.372313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f31c861aa9d4d508f676db9eea6d8fa'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:13.471071 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1251/2479] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:02:12 2022] DELETE /identity/v3/users/8f31c861aa9d4d508f676db9eea6d8fa => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:02:13.475713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441b171-5975-42b0-945c-3f49de4b8e83 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:13.476185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441b171-5975-42b0-945c-3f49de4b8e83 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:13.476317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441b171-5975-42b0-945c-3f49de4b8e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:13.493499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6441b171-5975-42b0-945c-3f49de4b8e83 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:13.494682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6441b171-5975-42b0-945c-3f49de4b8e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:13.551183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1229/2480] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:13.561298 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:13.562693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:13.625555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:13.637073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93fd68-dd58-40cd-82df-2d503f621727 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:13.641289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d70c9-0cdc-41ba-a935-ed2939c57039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:13.641972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d70c9-0cdc-41ba-a935-ed2939c57039 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:13.642252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d70c9-0cdc-41ba-a935-ed2939c57039 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:13.642531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d70c9-0cdc-41ba-a935-ed2939c57039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:13.655879 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d70c9-0cdc-41ba-a935-ed2939c57039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:13.657461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d70c9-0cdc-41ba-a935-ed2939c57039 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:13.670339 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1252/2481] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:02:14.194018 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441b171-5975-42b0-945c-3f49de4b8e83 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:14.195620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6441b171-5975-42b0-945c-3f49de4b8e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:14.218785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441b171-5975-42b0-945c-3f49de4b8e83 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:14.225334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:14.226218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:14.226574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:14.226932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] PATH_INFO: `/v3/projects/6857104436b34711b09ed7da81c5840d` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:14.228234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6857104436b34711b09ed7da81c5840d)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:14.230175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:14.335503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6857104436b34711b09ed7da81c5840d', 'request_id': 'req-7503ba12-7449-433b-93ac-6270a12a496d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:14.342632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6857104436b34711b09ed7da81c5840d', 'request_id': 'req-7503ba12-7449-433b-93ac-6270a12a496d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:14.348837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6857104436b34711b09ed7da81c5840d', 'request_id': 'req-7503ba12-7449-433b-93ac-6270a12a496d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:14.436281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] The token cache is being invalidate because project 6857104436b34711b09ed7da81c5840d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:02:14.436544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:14.437139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7503ba12-7449-433b-93ac-6270a12a496d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:14.438933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1230/2482] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:02:14 2022] DELETE /identity/v3/projects/6857104436b34711b09ed7da81c5840d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:02:14.445401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d38847-9d33-4ad0-b10a-c319a4cf74ae None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:14.446372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d38847-9d33-4ad0-b10a-c319a4cf74ae None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:14.446860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d38847-9d33-4ad0-b10a-c319a4cf74ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:14.471254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b0d38847-9d33-4ad0-b10a-c319a4cf74ae None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:14.472574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b0d38847-9d33-4ad0-b10a-c319a4cf74ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:14.533466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1253/2483] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:14.542284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb42144-0961-4ff1-a4ff-65f5bf697d44 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:14.542766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb42144-0961-4ff1-a4ff-65f5bf697d44 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:14.542872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb42144-0961-4ff1-a4ff-65f5bf697d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:14.560852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dbb42144-0961-4ff1-a4ff-65f5bf697d44 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:14.562049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dbb42144-0961-4ff1-a4ff-65f5bf697d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:14.570500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1231/2484] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:02:14.574520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d38847-9d33-4ad0-b10a-c319a4cf74ae None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:14.584908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d38847-9d33-4ad0-b10a-c319a4cf74ae None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:14.590013 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:14.590673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:14.590962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:14.591245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:14.592026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:02:14.592341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:14.593974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:14.605580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1254/2485] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:02:14.609698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb42144-0961-4ff1-a4ff-65f5bf697d44 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:14.618752 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb42144-0961-4ff1-a4ff-65f5bf697d44 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:14.623640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90fced-7488-4896-a06f-dcad5fba612e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:14.624284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90fced-7488-4896-a06f-dcad5fba612e admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:14.624570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90fced-7488-4896-a06f-dcad5fba612e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:14.624851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90fced-7488-4896-a06f-dcad5fba612e admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:14.625531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90fced-7488-4896-a06f-dcad5fba612e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:14.627523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90fced-7488-4896-a06f-dcad5fba612e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:14.750542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1232/2486] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:02:14 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:02:14.753113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:14.764875 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65620-46b6-4b4a-8e59-592172c80071 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:14.828446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36665bad-e454-4b14-8e0c-190f23e2c1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:14.828446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36665bad-e454-4b14-8e0c-190f23e2c1db admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:14.828446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36665bad-e454-4b14-8e0c-190f23e2c1db admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:14.828446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36665bad-e454-4b14-8e0c-190f23e2c1db admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:14.828950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36665bad-e454-4b14-8e0c-190f23e2c1db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:14.828950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36665bad-e454-4b14-8e0c-190f23e2c1db admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:14.935643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1255/2487] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:02:14 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:02:14.939853 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90fced-7488-4896-a06f-dcad5fba612e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:14.953149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90fced-7488-4896-a06f-dcad5fba612e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:14.959960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3026cf-28fe-4e0b-9fb7-b82f0039937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:14.960805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3026cf-28fe-4e0b-9fb7-b82f0039937a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:14.961163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3026cf-28fe-4e0b-9fb7-b82f0039937a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:14.961780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3026cf-28fe-4e0b-9fb7-b82f0039937a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:14.962811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3026cf-28fe-4e0b-9fb7-b82f0039937a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:14.964911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3026cf-28fe-4e0b-9fb7-b82f0039937a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:14.979931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1233/2488] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:02:14.984145 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36665bad-e454-4b14-8e0c-190f23e2c1db admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:14.999575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36665bad-e454-4b14-8e0c-190f23e2c1db admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:15.004739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d5be4-38f4-4369-9e91-f4d20d81b6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:15.005552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d5be4-38f4-4369-9e91-f4d20d81b6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:15.005840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d5be4-38f4-4369-9e91-f4d20d81b6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:15.006116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d5be4-38f4-4369-9e91-f4d20d81b6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:15.006851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d5be4-38f4-4369-9e91-f4d20d81b6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:15.009076 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d5be4-38f4-4369-9e91-f4d20d81b6b2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:15.026139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1256/2489] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:02:15.027129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3026cf-28fe-4e0b-9fb7-b82f0039937a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:15.050075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3026cf-28fe-4e0b-9fb7-b82f0039937a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:15.063286 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:15.064519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:15.064963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:15.065423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] PATH_INFO: `/v3/projects/2a5cf958fb8c4bbfae6f2de47a28b453/users/cc299dcbdb854321b72cf085d2f0d88a/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:15.098679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:15.102013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5cf958fb8c4bbfae6f2de47a28b453, user_id=cc299dcbdb854321b72cf085d2f0d88a, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:15.104879 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:15.174050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1234/2490] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:02:15 2022] PUT /identity/v3/projects/2a5cf958fb8c4bbfae6f2de47a28b453/users/cc299dcbdb854321b72cf085d2f0d88a/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:02:15.184749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a437335-7c4b-45a5-be27-6a9863cc8e12 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:15.185050 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a437335-7c4b-45a5-be27-6a9863cc8e12 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:15.185340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a437335-7c4b-45a5-be27-6a9863cc8e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:15.217210 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1a437335-7c4b-45a5-be27-6a9863cc8e12 None None] MFA Rules not processed for user `cc299dcbdb854321b72cf085d2f0d88a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:15.218874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1a437335-7c4b-45a5-be27-6a9863cc8e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:15.270952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:15.294785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1257/2491] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:15.306676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:15.308464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:15.366240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c5861-64fd-45bb-b575-f585e83be793 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:15.370046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f18d0f5f634d84a99fc3e46963a8b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:15.370697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:15.370988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:15.371271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:15.380412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:15.381858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:15.397391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1235/2492] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:02:25.386919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:25.413437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=8 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:35.001156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e794b3-cc9c-4009-b196-88882499e61e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:35.001701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e794b3-cc9c-4009-b196-88882499e61e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:35.001701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e794b3-cc9c-4009-b196-88882499e61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:35.019798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a1e794b3-cc9c-4009-b196-88882499e61e None None] MFA Rules not processed for user `a1daa10e8ac5455cbf98f02971639237`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:35.020999 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a1e794b3-cc9c-4009-b196-88882499e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:35.076751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1258/2493] 10.222.0.98 () {58 vars in 967 bytes} [Fri Jul 15 16:02:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:35.089134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:35.090563 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:35.169550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c756c2-3313-4349-b783-018de1673f2c service glance] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:35.174625 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155a961-819c-4f81-8b40-9f4b420503d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f69205a5fc042299a8913c924b45e45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:35.178883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155a961-819c-4f81-8b40-9f4b420503d5 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:35.178883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155a961-819c-4f81-8b40-9f4b420503d5 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:35.178883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155a961-819c-4f81-8b40-9f4b420503d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:35.184268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155a961-819c-4f81-8b40-9f4b420503d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:35.185872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155a961-819c-4f81-8b40-9f4b420503d5 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:35.194121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1236/2494] 10.222.0.98 () {60 vars in 1384 bytes} [Fri Jul 15 16:02:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 16:02:45.069209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=25 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:45.190830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=32 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:56.600022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e794b3-cc9c-4009-b196-88882499e61e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:56.663405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e794b3-cc9c-4009-b196-88882499e61e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:56.667964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:56.668851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:56.669138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:56.669419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] PATH_INFO: `/v3/users/cc299dcbdb854321b72cf085d2f0d88a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:56.670683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc299dcbdb854321b72cf085d2f0d88a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:56.672680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:56.907347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc299dcbdb854321b72cf085d2f0d88a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:56.919872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc299dcbdb854321b72cf085d2f0d88a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:56.920603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc299dcbdb854321b72cf085d2f0d88a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:56.921221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc299dcbdb854321b72cf085d2f0d88a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:56.928201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc299dcbdb854321b72cf085d2f0d88a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:57.005152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1259/2495] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:02:56 2022] DELETE /identity/v3/users/cc299dcbdb854321b72cf085d2f0d88a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:02:57.009746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1e51-d772-463f-b699-5676c00d5b37 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:57.010261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1e51-d772-463f-b699-5676c00d5b37 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:57.010368 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1e51-d772-463f-b699-5676c00d5b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:57.027653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-21fb1e51-d772-463f-b699-5676c00d5b37 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:57.028909 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-21fb1e51-d772-463f-b699-5676c00d5b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:57.088557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1237/2496] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:57.098304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:57.099844 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:57.160608 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb32a29-737d-4b0c-b326-fa7cceb3fd7b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:57.164829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac94a39-adb0-4ec1-b2ad-c3dc7624dd17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:57.165487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac94a39-adb0-4ec1-b2ad-c3dc7624dd17 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:57.165775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac94a39-adb0-4ec1-b2ad-c3dc7624dd17 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:57.166073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac94a39-adb0-4ec1-b2ad-c3dc7624dd17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:57.175680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac94a39-adb0-4ec1-b2ad-c3dc7624dd17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:57.177318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac94a39-adb0-4ec1-b2ad-c3dc7624dd17 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:57.188255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1260/2497] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:02:57.637200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb1e51-d772-463f-b699-5676c00d5b37 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:57.638852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-21fb1e51-d772-463f-b699-5676c00d5b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:57.654663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb1e51-d772-463f-b699-5676c00d5b37 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:57.659592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:57.660477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:57.660764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:57.661044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] PATH_INFO: `/v3/projects/2a5cf958fb8c4bbfae6f2de47a28b453` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:57.663134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5cf958fb8c4bbfae6f2de47a28b453)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:57.665680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:57.785297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5cf958fb8c4bbfae6f2de47a28b453', 'request_id': 'req-d6c274dc-f659-47fa-aa2e-cff509c40682'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:57.791595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5cf958fb8c4bbfae6f2de47a28b453', 'request_id': 'req-d6c274dc-f659-47fa-aa2e-cff509c40682'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:57.797252 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5cf958fb8c4bbfae6f2de47a28b453', 'request_id': 'req-d6c274dc-f659-47fa-aa2e-cff509c40682'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:57.875483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] The token cache is being invalidate because project 2a5cf958fb8c4bbfae6f2de47a28b453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:02:57.875782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:57.876415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d6c274dc-f659-47fa-aa2e-cff509c40682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:02:57.877984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1238/2498] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:02:57 2022] DELETE /identity/v3/projects/2a5cf958fb8c4bbfae6f2de47a28b453 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:02:57.887679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1f24b-7e62-470a-b397-b1bd6cfdc39f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:57.887679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1f24b-7e62-470a-b397-b1bd6cfdc39f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:57.887679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1f24b-7e62-470a-b397-b1bd6cfdc39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:57.902087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-13e1f24b-7e62-470a-b397-b1bd6cfdc39f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:57.903338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-13e1f24b-7e62-470a-b397-b1bd6cfdc39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:57.919545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:57.960055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1261/2499] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:57.969858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc06167-02a0-4556-a3a2-2357fc766f31 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:57.970086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc06167-02a0-4556-a3a2-2357fc766f31 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:57.970362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc06167-02a0-4556-a3a2-2357fc766f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:57.989211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ffc06167-02a0-4556-a3a2-2357fc766f31 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:57.992230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ffc06167-02a0-4556-a3a2-2357fc766f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:57.998520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1239/2500] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:02:58.002868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1f24b-7e62-470a-b397-b1bd6cfdc39f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:58.013924 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1f24b-7e62-470a-b397-b1bd6cfdc39f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:58.018441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:58.020914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:58.020914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:58.020914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:58.020914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:02:58.020914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:58.022236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:58.034195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1262/2501] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:02:58.038461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc06167-02a0-4556-a3a2-2357fc766f31 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:58.048150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc06167-02a0-4556-a3a2-2357fc766f31 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:58.052818 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18b742-ea3a-4d5e-b495-271b07d29da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:58.053470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18b742-ea3a-4d5e-b495-271b07d29da2 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:58.053760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18b742-ea3a-4d5e-b495-271b07d29da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:58.054042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18b742-ea3a-4d5e-b495-271b07d29da2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:58.054737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18b742-ea3a-4d5e-b495-271b07d29da2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:58.057279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18b742-ea3a-4d5e-b495-271b07d29da2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:58.424748 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1240/2502] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:02:58 2022] POST /identity/v3/projects => generated 379 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:02:58.429131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:58.439018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2e30f-36be-4b03-af9a-c101862c2b22 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:58.549087 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b69d5-8e4b-4469-95b5-341caf22753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:58.549692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b69d5-8e4b-4469-95b5-341caf22753c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:58.549982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b69d5-8e4b-4469-95b5-341caf22753c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:58.550263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b69d5-8e4b-4469-95b5-341caf22753c admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:58.550979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b69d5-8e4b-4469-95b5-341caf22753c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:58.552947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b69d5-8e4b-4469-95b5-341caf22753c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:58.699696 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1263/2503] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:02:58 2022] POST /identity/v3/users => generated 290 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:02:58.713058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18b742-ea3a-4d5e-b495-271b07d29da2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:58.723348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18b742-ea3a-4d5e-b495-271b07d29da2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:58.727900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502f32c-7ae8-410b-bdfc-193afcd7fbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:58.728553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502f32c-7ae8-410b-bdfc-193afcd7fbfb admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:58.728840 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502f32c-7ae8-410b-bdfc-193afcd7fbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:58.729121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502f32c-7ae8-410b-bdfc-193afcd7fbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:58.729852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502f32c-7ae8-410b-bdfc-193afcd7fbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:58.731479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502f32c-7ae8-410b-bdfc-193afcd7fbfb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:58.741754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1241/2504] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:02:58.745847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b69d5-8e4b-4469-95b5-341caf22753c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:58.755775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b69d5-8e4b-4469-95b5-341caf22753c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:58.760910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fffeef-ad77-45f9-8901-61174490ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:58.761550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffeef-ad77-45f9-8901-61174490ac84 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:58.761862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffeef-ad77-45f9-8901-61174490ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:58.762144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffeef-ad77-45f9-8901-61174490ac84 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:58.762858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffeef-ad77-45f9-8901-61174490ac84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:58.764676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffeef-ad77-45f9-8901-61174490ac84 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:58.775129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1264/2505] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:02:58.779146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502f32c-7ae8-410b-bdfc-193afcd7fbfb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:58.789139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502f32c-7ae8-410b-bdfc-193afcd7fbfb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:58.793951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:58.794835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:58.795124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:58.795452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] PATH_INFO: `/v3/projects/836bc4ea37f841cf9704999d88a627c4/users/cd49fd5c3cd5463b88efa5937d88cea0/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:58.810421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] RBAC: Authorizing `identity:create_grant(project_id=836bc4ea37f841cf9704999d88a627c4, user_id=cd49fd5c3cd5463b88efa5937d88cea0, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:58.813129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:58.937718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1242/2506] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:02:58 2022] PUT /identity/v3/projects/836bc4ea37f841cf9704999d88a627c4/users/cd49fd5c3cd5463b88efa5937d88cea0/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:02:58.942766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e89dea-2697-4191-8241-c27c89d16814 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:58.943266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e89dea-2697-4191-8241-c27c89d16814 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:58.943418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e89dea-2697-4191-8241-c27c89d16814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:58.961462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c6e89dea-2697-4191-8241-c27c89d16814 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:02:58.962931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c6e89dea-2697-4191-8241-c27c89d16814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:59.020781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1265/2507] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:02:59.030753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:02:59.032564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:02:59.054156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:02:59.090595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c569b-addd-4447-9b1a-b43a85a4260c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:02:59.094317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:02:59.094977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:02:59.095264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:02:59.095603 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:02:59.104883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:02:59.106342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:02:59.116777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1243/2508] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:00.547682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03dd89-7a1a-4003-a5f2-af18719691f7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:00.551452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03dd89-7a1a-4003-a5f2-af18719691f7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:00.551452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03dd89-7a1a-4003-a5f2-af18719691f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:00.576033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5f03dd89-7a1a-4003-a5f2-af18719691f7 None None] MFA Rules not processed for user `cd49fd5c3cd5463b88efa5937d88cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:00.577358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5f03dd89-7a1a-4003-a5f2-af18719691f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:00.594617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:00.642068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1266/2509] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:00.652776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:00.654192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:00.712026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecc1b-45bf-46d3-a7f4-b47c4df3a20b service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:00.715983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41186ccd-9132-461e-931f-03cc9fc63edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:00.716630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41186ccd-9132-461e-931f-03cc9fc63edb service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:00.716924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41186ccd-9132-461e-931f-03cc9fc63edb service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:00.717210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41186ccd-9132-461e-931f-03cc9fc63edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:00.725459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41186ccd-9132-461e-931f-03cc9fc63edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:00.727497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41186ccd-9132-461e-931f-03cc9fc63edb service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:00.736327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1244/2510] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:02.488945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05197813-d822-43b0-8804-e007eab01202 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:02.489468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05197813-d822-43b0-8804-e007eab01202 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:02.489562 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05197813-d822-43b0-8804-e007eab01202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:02.507125 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-05197813-d822-43b0-8804-e007eab01202 None None] MFA Rules not processed for user `cd49fd5c3cd5463b88efa5937d88cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:02.508770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-05197813-d822-43b0-8804-e007eab01202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:02.515558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1267/2511] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:02.769392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41186ccd-9132-461e-931f-03cc9fc63edb service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:02.783022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41186ccd-9132-461e-931f-03cc9fc63edb service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:02.786631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:02.787274 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:02.787590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:02.787869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:02.790347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:02.804349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:02.805844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:02.814344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1245/2512] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:12.501994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=22 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:12.810459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=49 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:27.658977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05197813-d822-43b0-8804-e007eab01202 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:27.666446 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-05197813-d822-43b0-8804-e007eab01202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:27.673451 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05197813-d822-43b0-8804-e007eab01202 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:27.677088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:27.677755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:27.678062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:27.678349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:27.680194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:27.694077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:27.696051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:27.707574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1268/2513] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:31.500186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:31.501740 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:31.579981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebec874-8d1e-402a-914f-e253e06067a9 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:31.592174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:31.592174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:31.592174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:31.592174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:31.592606 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:31.600204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:31.601800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:31.612396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1246/2514] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:31.623827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:31.626723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:31.639248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ab058-62e9-40fe-8432-7e4da1d437ed service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:31.644062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:31.644945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:31.645232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:31.645513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] PATH_INFO: `/v3/users/78db558bb3ea4161b4a54e62f9ec3bd6` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:31.661089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78db558bb3ea4161b4a54e62f9ec3bd6)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:31.663580 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:31.839751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78db558bb3ea4161b4a54e62f9ec3bd6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:31.852263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78db558bb3ea4161b4a54e62f9ec3bd6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:31.853099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78db558bb3ea4161b4a54e62f9ec3bd6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:31.853771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78db558bb3ea4161b4a54e62f9ec3bd6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:31.861375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ae0a9e6e-0f3f-4f0c-89fe-868631648d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78db558bb3ea4161b4a54e62f9ec3bd6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:31.971985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1269/2515] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:03:31 2022] DELETE /identity/v3/users/78db558bb3ea4161b4a54e62f9ec3bd6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:32.238776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:32.249587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2276d5-3a8c-4e77-aa42-4d685635c115 service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:32.314376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:32.315236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:32.315560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:32.315835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] PATH_INFO: `/v3/projects/93e4d5ff424c4e4fa5dd81c20cc920d0` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:32.327779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e4d5ff424c4e4fa5dd81c20cc920d0)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:32.329715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:32.478757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e4d5ff424c4e4fa5dd81c20cc920d0', 'request_id': 'req-679b2724-502e-4ec1-9aaf-52bdf93beee3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:32.485539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e4d5ff424c4e4fa5dd81c20cc920d0', 'request_id': 'req-679b2724-502e-4ec1-9aaf-52bdf93beee3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:32.491100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e4d5ff424c4e4fa5dd81c20cc920d0', 'request_id': 'req-679b2724-502e-4ec1-9aaf-52bdf93beee3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:32.590092 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] The token cache is being invalidate because project 93e4d5ff424c4e4fa5dd81c20cc920d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:03:32.590821 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:32.591088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-679b2724-502e-4ec1-9aaf-52bdf93beee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:32.592809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1247/2516] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:03:32 2022] DELETE /identity/v3/projects/93e4d5ff424c4e4fa5dd81c20cc920d0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:32.597850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa4b7-69dc-4bbf-b9da-0e0a574bc61f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:32.598147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa4b7-69dc-4bbf-b9da-0e0a574bc61f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:32.602954 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966aa4b7-69dc-4bbf-b9da-0e0a574bc61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:32.617128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-966aa4b7-69dc-4bbf-b9da-0e0a574bc61f None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:32.618900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-966aa4b7-69dc-4bbf-b9da-0e0a574bc61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:32.683806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1270/2517] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:32.689067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac849de8-eb0d-49b3-b1fa-d15261a596f7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:32.689363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac849de8-eb0d-49b3-b1fa-d15261a596f7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:32.689646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac849de8-eb0d-49b3-b1fa-d15261a596f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:32.708090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ac849de8-eb0d-49b3-b1fa-d15261a596f7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:32.709317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ac849de8-eb0d-49b3-b1fa-d15261a596f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:32.717566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1248/2518] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:32.721872 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aa4b7-69dc-4bbf-b9da-0e0a574bc61f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:32.732085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966aa4b7-69dc-4bbf-b9da-0e0a574bc61f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:32.736798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:32.737462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:32.737752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:32.738036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:32.738910 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:03:32.739212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:32.740857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:32.753106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1271/2519] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:03:32.757508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac849de8-eb0d-49b3-b1fa-d15261a596f7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:32.764504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:32.768394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac849de8-eb0d-49b3-b1fa-d15261a596f7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:32.773108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62b41-1299-4110-af92-ec9da9fb3ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:32.773801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62b41-1299-4110-af92-ec9da9fb3ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:32.774084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62b41-1299-4110-af92-ec9da9fb3ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:32.774361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62b41-1299-4110-af92-ec9da9fb3ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:32.775065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae62b41-1299-4110-af92-ec9da9fb3ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:32.777025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae62b41-1299-4110-af92-ec9da9fb3ca7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:32.915037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1249/2520] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:03:32 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:03:32.916851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:32.927892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7dda4-a857-4ba0-95f6-235d6037a1d7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:32.962701 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:32.991267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95885eeb-21ec-4641-8998-c6330de0b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:32.991980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95885eeb-21ec-4641-8998-c6330de0b0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:32.992266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95885eeb-21ec-4641-8998-c6330de0b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:32.992549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95885eeb-21ec-4641-8998-c6330de0b0d3 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:32.993261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95885eeb-21ec-4641-8998-c6330de0b0d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:32.995761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95885eeb-21ec-4641-8998-c6330de0b0d3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.105375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1272/2521] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:03:32 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:03:33.109559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62b41-1299-4110-af92-ec9da9fb3ca7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:33.120088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62b41-1299-4110-af92-ec9da9fb3ca7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:33.124729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a3730-200e-43dd-b1f5-76c51d7fbe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:33.125387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3730-200e-43dd-b1f5-76c51d7fbe50 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.125726 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3730-200e-43dd-b1f5-76c51d7fbe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.126009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a3730-200e-43dd-b1f5-76c51d7fbe50 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.126725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a3730-200e-43dd-b1f5-76c51d7fbe50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:33.128370 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a3730-200e-43dd-b1f5-76c51d7fbe50 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.140295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1250/2522] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:33.143635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95885eeb-21ec-4641-8998-c6330de0b0d3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:33.154327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95885eeb-21ec-4641-8998-c6330de0b0d3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:33.159220 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:33.166036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.166036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.166036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.166036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:33.166036 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.174797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1273/2523] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:33.178925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a3730-200e-43dd-b1f5-76c51d7fbe50 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:33.189213 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a3730-200e-43dd-b1f5-76c51d7fbe50 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:33.194005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d12a87-4abe-4fe8-b0c4-74fa054028f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:33.194970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12a87-4abe-4fe8-b0c4-74fa054028f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.195261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12a87-4abe-4fe8-b0c4-74fa054028f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.195574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d12a87-4abe-4fe8-b0c4-74fa054028f7 admin admin] PATH_INFO: `/v3/projects/abe1a50a1d694a4fb698b8d47932603f/users/8734b3a1739f4e13a2c354b16d73ad73/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.217620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d12a87-4abe-4fe8-b0c4-74fa054028f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe1a50a1d694a4fb698b8d47932603f, user_id=8734b3a1739f4e13a2c354b16d73ad73, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:33.217620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d12a87-4abe-4fe8-b0c4-74fa054028f7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.294392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:33.296371 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:33.319929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1251/2524] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:03:33 2022] PUT /identity/v3/projects/abe1a50a1d694a4fb698b8d47932603f/users/8734b3a1739f4e13a2c354b16d73ad73/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:33.330351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320556d-8d9c-4644-9964-20bc29cc41ad None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.330730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320556d-8d9c-4644-9964-20bc29cc41ad None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.331117 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320556d-8d9c-4644-9964-20bc29cc41ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.358810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5320556d-8d9c-4644-9964-20bc29cc41ad None None] MFA Rules not processed for user `8734b3a1739f4e13a2c354b16d73ad73`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:33.360088 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5320556d-8d9c-4644-9964-20bc29cc41ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:33.360390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39f82a-8c66-4bbc-be56-e0311108c71c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:33.420290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:33.421231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.421525 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.421806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.423748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:33.450495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1252/2525] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:03:33.455161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320556d-8d9c-4644-9964-20bc29cc41ad None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:33.504193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:33.505560 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.518232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1274/2526] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:03:33.522746 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320556d-8d9c-4644-9964-20bc29cc41ad None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:33.527548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3403886-86c2-4686-a10e-454ad142ddb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:33.528223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403886-86c2-4686-a10e-454ad142ddb0 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.528513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403886-86c2-4686-a10e-454ad142ddb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.528794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403886-86c2-4686-a10e-454ad142ddb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.529481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3403886-86c2-4686-a10e-454ad142ddb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:33.531430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3403886-86c2-4686-a10e-454ad142ddb0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.616755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1253/2527] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:03:33 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:03:33.621434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:33.636147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb62a90-bf75-4546-a42a-bf9ee15a6286 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:33.644761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:33.694617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e91f68-f043-4508-beac-156ce832fc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:33.695277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e91f68-f043-4508-beac-156ce832fc30 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.695597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e91f68-f043-4508-beac-156ce832fc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.695876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e91f68-f043-4508-beac-156ce832fc30 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.696684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e91f68-f043-4508-beac-156ce832fc30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:33.698635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e91f68-f043-4508-beac-156ce832fc30 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.812435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1275/2528] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:03:33 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:03:33.816508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3403886-86c2-4686-a10e-454ad142ddb0 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:33.827264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3403886-86c2-4686-a10e-454ad142ddb0 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:33.832098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47f755-b673-4624-93e7-bbe3124d478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:33.832783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47f755-b673-4624-93e7-bbe3124d478c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.833073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47f755-b673-4624-93e7-bbe3124d478c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.833357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47f755-b673-4624-93e7-bbe3124d478c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.834090 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47f755-b673-4624-93e7-bbe3124d478c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:33.835776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47f755-b673-4624-93e7-bbe3124d478c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.847786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1254/2529] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:33.852568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e91f68-f043-4508-beac-156ce832fc30 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:33.863383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e91f68-f043-4508-beac-156ce832fc30 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:33.867951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fab3e-e35f-4d32-b915-c20e18bf40d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:33.869461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fab3e-e35f-4d32-b915-c20e18bf40d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:33.869752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fab3e-e35f-4d32-b915-c20e18bf40d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:33.870083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fab3e-e35f-4d32-b915-c20e18bf40d3 admin admin] PATH_INFO: `/v3/projects/ee662700a411404285627729d1fbd756/users/12a133b5c5404da1a938eaa5cd5d1d8b/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:33.886237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fab3e-e35f-4d32-b915-c20e18bf40d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee662700a411404285627729d1fbd756, user_id=12a133b5c5404da1a938eaa5cd5d1d8b, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:33.888964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fab3e-e35f-4d32-b915-c20e18bf40d3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:33.996341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1276/2530] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:03:33 2022] PUT /identity/v3/projects/ee662700a411404285627729d1fbd756/users/12a133b5c5404da1a938eaa5cd5d1d8b/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:34.000258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47f755-b673-4624-93e7-bbe3124d478c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:34.005476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:34.011239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47f755-b673-4624-93e7-bbe3124d478c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:34.068892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ae733-b58f-40a7-a150-be4f306e46a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:34.069504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ae733-b58f-40a7-a150-be4f306e46a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:34.069790 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ae733-b58f-40a7-a150-be4f306e46a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:34.070069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ae733-b58f-40a7-a150-be4f306e46a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:34.070799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ae733-b58f-40a7-a150-be4f306e46a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:34.073237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ae733-b58f-40a7-a150-be4f306e46a5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:34.089294 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1255/2531] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:34.093323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fab3e-e35f-4d32-b915-c20e18bf40d3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:34.104025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fab3e-e35f-4d32-b915-c20e18bf40d3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:34.108862 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:34.109753 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:34.110046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:34.110317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] PATH_INFO: `/v3/projects/ee662700a411404285627729d1fbd756/users/12a133b5c5404da1a938eaa5cd5d1d8b/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:34.112397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee662700a411404285627729d1fbd756, user_id=12a133b5c5404da1a938eaa5cd5d1d8b, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:34.115139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:34.127826 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1277/2532] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:03:34 2022] PUT /identity/v3/projects/ee662700a411404285627729d1fbd756/users/12a133b5c5404da1a938eaa5cd5d1d8b/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:03:34.139444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9dce0-27d6-4ceb-a5ee-a107568b75dd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:34.139767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9dce0-27d6-4ceb-a5ee-a107568b75dd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:34.140053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9dce0-27d6-4ceb-a5ee-a107568b75dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:34.168702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f0f9dce0-27d6-4ceb-a5ee-a107568b75dd None None] MFA Rules not processed for user `12a133b5c5404da1a938eaa5cd5d1d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:34.169930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f0f9dce0-27d6-4ceb-a5ee-a107568b75dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:34.241162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1256/2533] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:34.253501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:34.255198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:34.315448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3f298-0604-4354-8c10-5fd91454b77f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:34.319164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5158afe-1520-46c9-9e0e-83730396eab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:34.319785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5158afe-1520-46c9-9e0e-83730396eab2 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:34.320072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5158afe-1520-46c9-9e0e-83730396eab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:34.320405 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5158afe-1520-46c9-9e0e-83730396eab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:34.329959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5158afe-1520-46c9-9e0e-83730396eab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:34.331418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5158afe-1520-46c9-9e0e-83730396eab2 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:34.341156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1278/2534] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:34.696416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9dce0-27d6-4ceb-a5ee-a107568b75dd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:34.697885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f0f9dce0-27d6-4ceb-a5ee-a107568b75dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:34.756937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9dce0-27d6-4ceb-a5ee-a107568b75dd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:34.760592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08128c-a0bc-44b6-8f20-62931d66e659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:34.761222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08128c-a0bc-44b6-8f20-62931d66e659 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:34.761507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08128c-a0bc-44b6-8f20-62931d66e659 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:34.761792 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08128c-a0bc-44b6-8f20-62931d66e659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:34.763580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5e08128c-a0bc-44b6-8f20-62931d66e659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:34.802142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:34.835124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08128c-a0bc-44b6-8f20-62931d66e659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:34.835124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08128c-a0bc-44b6-8f20-62931d66e659 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:34.847706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1257/2535] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:35.653389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5158afe-1520-46c9-9e0e-83730396eab2 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:35.718023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5158afe-1520-46c9-9e0e-83730396eab2 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:35.723422 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:35.728545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:35.728545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:35.728545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] PATH_INFO: `/v3/users/8734b3a1739f4e13a2c354b16d73ad73` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:35.728545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8734b3a1739f4e13a2c354b16d73ad73)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:35.728972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:35.734285 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:35.932922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8734b3a1739f4e13a2c354b16d73ad73'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:35.944845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8734b3a1739f4e13a2c354b16d73ad73'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:35.945551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8734b3a1739f4e13a2c354b16d73ad73'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:35.946139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8734b3a1739f4e13a2c354b16d73ad73'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:35.953397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8734b3a1739f4e13a2c354b16d73ad73'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:36.098847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1279/2536] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:03:35 2022] DELETE /identity/v3/users/8734b3a1739f4e13a2c354b16d73ad73 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:36.103447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceed74-d253-423c-9168-2bca54414358 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:36.103939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceed74-d253-423c-9168-2bca54414358 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:36.104073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceed74-d253-423c-9168-2bca54414358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:36.121204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fcceed74-d253-423c-9168-2bca54414358 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:36.122490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fcceed74-d253-423c-9168-2bca54414358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:36.191839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1258/2537] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:03:36.200373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:36.201889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:36.265501 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d6cf0-3c41-4f9b-abb9-d3510b6cdf09 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:36.269169 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b772d-57ef-4b7e-a47a-6a82289477e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:36.269816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b772d-57ef-4b7e-a47a-6a82289477e8 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:36.270100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b772d-57ef-4b7e-a47a-6a82289477e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:36.270376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b772d-57ef-4b7e-a47a-6a82289477e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:36.280923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b772d-57ef-4b7e-a47a-6a82289477e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:36.282500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b772d-57ef-4b7e-a47a-6a82289477e8 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:36.292530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1280/2538] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:36.756202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceed74-d253-423c-9168-2bca54414358 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:36.757928 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fcceed74-d253-423c-9168-2bca54414358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:36.782315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceed74-d253-423c-9168-2bca54414358 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:36.807653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:36.807653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:36.807653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:36.807653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] PATH_INFO: `/v3/projects/abe1a50a1d694a4fb698b8d47932603f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:36.808449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe1a50a1d694a4fb698b8d47932603f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:36.808449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:36.973831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe1a50a1d694a4fb698b8d47932603f', 'request_id': 'req-2c656d44-7efc-4d60-83b2-0f7baf082acd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:36.980391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe1a50a1d694a4fb698b8d47932603f', 'request_id': 'req-2c656d44-7efc-4d60-83b2-0f7baf082acd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:36.986673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe1a50a1d694a4fb698b8d47932603f', 'request_id': 'req-2c656d44-7efc-4d60-83b2-0f7baf082acd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:37.087354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] The token cache is being invalidate because project abe1a50a1d694a4fb698b8d47932603f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:03:37.088405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:37.089233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:37.090851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1259/2539] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:03:36 2022] DELETE /identity/v3/projects/abe1a50a1d694a4fb698b8d47932603f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:37.094992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b772d-57ef-4b7e-a47a-6a82289477e8 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:37.105069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b772d-57ef-4b7e-a47a-6a82289477e8 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:37.167755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:37.168726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:37.169080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:37.169430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] PATH_INFO: `/v3/users/12a133b5c5404da1a938eaa5cd5d1d8b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:37.170741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a133b5c5404da1a938eaa5cd5d1d8b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:37.172806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:37.463449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:37.476748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a133b5c5404da1a938eaa5cd5d1d8b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:37.489717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a133b5c5404da1a938eaa5cd5d1d8b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:37.490432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a133b5c5404da1a938eaa5cd5d1d8b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:37.491053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a133b5c5404da1a938eaa5cd5d1d8b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:37.498758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-83508ef0-0f02-4a67-bf6c-1270b969a939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a133b5c5404da1a938eaa5cd5d1d8b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:37.599651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1281/2540] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:03:37 2022] DELETE /identity/v3/users/12a133b5c5404da1a938eaa5cd5d1d8b => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:38.123975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:38.125440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:38.139542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656d44-7efc-4d60-83b2-0f7baf082acd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:38.185430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:38.197086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:38.197941 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:38.198235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:38.198513 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] PATH_INFO: `/v3/projects/ee662700a411404285627729d1fbd756` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:38.199766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee662700a411404285627729d1fbd756)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:38.201738 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:38.355382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee662700a411404285627729d1fbd756', 'request_id': 'req-92bea5e1-b10c-4214-8c6f-56839292438e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:38.361580 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee662700a411404285627729d1fbd756', 'request_id': 'req-92bea5e1-b10c-4214-8c6f-56839292438e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:38.367161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee662700a411404285627729d1fbd756', 'request_id': 'req-92bea5e1-b10c-4214-8c6f-56839292438e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:38.481327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] The token cache is being invalidate because project ee662700a411404285627729d1fbd756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:03:38.481620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:38.482237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-92bea5e1-b10c-4214-8c6f-56839292438e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:38.483835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1260/2541] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:03:38 2022] DELETE /identity/v3/projects/ee662700a411404285627729d1fbd756 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:38.488848 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e97f3-ffae-4f2d-93c4-05952f91487d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:38.489323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e97f3-ffae-4f2d-93c4-05952f91487d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:38.489545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e97f3-ffae-4f2d-93c4-05952f91487d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:38.509516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3d2e97f3-ffae-4f2d-93c4-05952f91487d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:38.510669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d2e97f3-ffae-4f2d-93c4-05952f91487d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:38.573154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1282/2542] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:38.581061 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf5c7-2cfb-4c93-a2eb-3a778f9f0ecf None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:38.581356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf5c7-2cfb-4c93-a2eb-3a778f9f0ecf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:38.581622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cf5c7-2cfb-4c93-a2eb-3a778f9f0ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:38.600754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ff1cf5c7-2cfb-4c93-a2eb-3a778f9f0ecf None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:38.601976 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ff1cf5c7-2cfb-4c93-a2eb-3a778f9f0ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:38.610762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1261/2543] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:38.615002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e97f3-ffae-4f2d-93c4-05952f91487d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:38.625113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e97f3-ffae-4f2d-93c4-05952f91487d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:38.629833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:38.630505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:38.630795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:38.631079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:38.631847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:03:38.632160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:38.633793 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:38.647348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1283/2544] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:03:38.650998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cf5c7-2cfb-4c93-a2eb-3a778f9f0ecf None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:38.660225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cf5c7-2cfb-4c93-a2eb-3a778f9f0ecf None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:38.664829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331dbd3-4d44-4f35-aa2a-5a0b46f7b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:38.666063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331dbd3-4d44-4f35-aa2a-5a0b46f7b0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:38.666351 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331dbd3-4d44-4f35-aa2a-5a0b46f7b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:38.666633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331dbd3-4d44-4f35-aa2a-5a0b46f7b0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:38.667355 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331dbd3-4d44-4f35-aa2a-5a0b46f7b0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:38.669325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331dbd3-4d44-4f35-aa2a-5a0b46f7b0b7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:38.821767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1262/2545] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:03:38 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:03:38.824141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:38.834332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacc0b7-cf5f-42b9-b7ce-3f4343cc8949 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:38.895542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6e8e8-67af-4219-b6d1-6f249afa6b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:38.896221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e8e8-67af-4219-b6d1-6f249afa6b2e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:38.896503 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e8e8-67af-4219-b6d1-6f249afa6b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:38.896780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e8e8-67af-4219-b6d1-6f249afa6b2e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:38.897471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6e8e8-67af-4219-b6d1-6f249afa6b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:38.899403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6e8e8-67af-4219-b6d1-6f249afa6b2e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:39.011383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1284/2546] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:03:38 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:03:39.012745 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331dbd3-4d44-4f35-aa2a-5a0b46f7b0b7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:39.023548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331dbd3-4d44-4f35-aa2a-5a0b46f7b0b7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:39.028933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5573ea1-5a25-4e59-b6de-bfeade04b6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:39.029605 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5573ea1-5a25-4e59-b6de-bfeade04b6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:39.029897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5573ea1-5a25-4e59-b6de-bfeade04b6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:39.030176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5573ea1-5a25-4e59-b6de-bfeade04b6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:39.030908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5573ea1-5a25-4e59-b6de-bfeade04b6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:39.032660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5573ea1-5a25-4e59-b6de-bfeade04b6e1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:39.045955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1263/2547] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:39.047163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6e8e8-67af-4219-b6d1-6f249afa6b2e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:39.057517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6e8e8-67af-4219-b6d1-6f249afa6b2e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:39.062442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165a112-99a2-4c87-92a5-28347d692b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:39.063043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a112-99a2-4c87-92a5-28347d692b00 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:39.063360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a112-99a2-4c87-92a5-28347d692b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:39.063675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a112-99a2-4c87-92a5-28347d692b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:39.065507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165a112-99a2-4c87-92a5-28347d692b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:39.067130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165a112-99a2-4c87-92a5-28347d692b00 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:39.078592 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1285/2548] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:03:39.082908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5573ea1-5a25-4e59-b6de-bfeade04b6e1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:39.093969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5573ea1-5a25-4e59-b6de-bfeade04b6e1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:39.098596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:39.099791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:39.099791 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:39.100068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] PATH_INFO: `/v3/projects/2aebb19f9c6444fbbe7420b90cd3709e/users/f15e8186fd88444994c75043dd09a0f3/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:39.115639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aebb19f9c6444fbbe7420b90cd3709e, user_id=f15e8186fd88444994c75043dd09a0f3, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:39.118470 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:39.243757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1264/2549] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:03:39 2022] PUT /identity/v3/projects/2aebb19f9c6444fbbe7420b90cd3709e/users/f15e8186fd88444994c75043dd09a0f3/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:39.250905 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9431226-8d06-4629-86bd-f8e95da03974 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:39.251206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9431226-8d06-4629-86bd-f8e95da03974 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:39.251532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9431226-8d06-4629-86bd-f8e95da03974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:39.270223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c9431226-8d06-4629-86bd-f8e95da03974 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:39.271491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c9431226-8d06-4629-86bd-f8e95da03974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:39.281072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:39.341468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1286/2550] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:39.351796 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:39.353594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:39.418715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8879f0-371e-4131-84b2-12462dd89da3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:39.422454 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d56b4-3a96-4eec-b8c8-3df1eb3decc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:39.423106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d56b4-3a96-4eec-b8c8-3df1eb3decc9 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:39.423420 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d56b4-3a96-4eec-b8c8-3df1eb3decc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:39.423702 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d56b4-3a96-4eec-b8c8-3df1eb3decc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:39.433151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d56b4-3a96-4eec-b8c8-3df1eb3decc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:39.439162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d56b4-3a96-4eec-b8c8-3df1eb3decc9 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:39.444114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1265/2551] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:41.001411 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef346e-8e6f-4b2d-97ce-dedb3913cd64 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:41.014947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef346e-8e6f-4b2d-97ce-dedb3913cd64 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:41.014947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef346e-8e6f-4b2d-97ce-dedb3913cd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:41.032252 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e3ef346e-8e6f-4b2d-97ce-dedb3913cd64 None None] MFA Rules not processed for user `f15e8186fd88444994c75043dd09a0f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:41.033923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e3ef346e-8e6f-4b2d-97ce-dedb3913cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:41.105909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1287/2552] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:41.110468 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d56b4-3a96-4eec-b8c8-3df1eb3decc9 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:41.120516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d56b4-3a96-4eec-b8c8-3df1eb3decc9 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:41.125235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388af691-369c-46bd-a646-3e083452466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:41.125897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388af691-369c-46bd-a646-3e083452466e admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:41.126186 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388af691-369c-46bd-a646-3e083452466e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:41.126466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388af691-369c-46bd-a646-3e083452466e admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:41.127166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388af691-369c-46bd-a646-3e083452466e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:41.129120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388af691-369c-46bd-a646-3e083452466e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:41.143923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:41.346724 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1266/2553] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:03:41 2022] POST /identity/v3/projects => generated 371 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:03:41.350960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef346e-8e6f-4b2d-97ce-dedb3913cd64 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:41.362017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef346e-8e6f-4b2d-97ce-dedb3913cd64 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:41.419635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59467ead-9523-4e24-94d2-6438f16b251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:41.420320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59467ead-9523-4e24-94d2-6438f16b251a admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:41.420611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59467ead-9523-4e24-94d2-6438f16b251a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:41.420890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59467ead-9523-4e24-94d2-6438f16b251a admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:41.421594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59467ead-9523-4e24-94d2-6438f16b251a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:41.423692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59467ead-9523-4e24-94d2-6438f16b251a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:41.675850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1288/2554] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:03:41 2022] POST /identity/v3/users => generated 292 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:03:41.679882 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388af691-369c-46bd-a646-3e083452466e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:41.690403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388af691-369c-46bd-a646-3e083452466e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:41.694930 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7f0b-8d21-4911-9c0f-f56652e689bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:41.695602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7f0b-8d21-4911-9c0f-f56652e689bf admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:41.695891 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7f0b-8d21-4911-9c0f-f56652e689bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:41.696165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a7f0b-8d21-4911-9c0f-f56652e689bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:41.696878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a7f0b-8d21-4911-9c0f-f56652e689bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:41.699316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a7f0b-8d21-4911-9c0f-f56652e689bf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:41.712689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1267/2555] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:41.717086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59467ead-9523-4e24-94d2-6438f16b251a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:41.727365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59467ead-9523-4e24-94d2-6438f16b251a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:41.732476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25257992-d435-485a-9b13-756b0b883c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:41.733357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25257992-d435-485a-9b13-756b0b883c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:41.733644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25257992-d435-485a-9b13-756b0b883c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:41.733922 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25257992-d435-485a-9b13-756b0b883c7d admin admin] PATH_INFO: `/v3/projects/e1b4f6e21e4a404e87c04ee5ac394ca6/users/9292de923f3b49b48503ca933e93bde4/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:41.749544 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25257992-d435-485a-9b13-756b0b883c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b4f6e21e4a404e87c04ee5ac394ca6, user_id=9292de923f3b49b48503ca933e93bde4, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:41.752474 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25257992-d435-485a-9b13-756b0b883c7d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:41.862725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1289/2556] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:03:41 2022] PUT /identity/v3/projects/e1b4f6e21e4a404e87c04ee5ac394ca6/users/9292de923f3b49b48503ca933e93bde4/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:03:41.866672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7f0b-8d21-4911-9c0f-f56652e689bf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:41.876825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a7f0b-8d21-4911-9c0f-f56652e689bf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:41.934566 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bac13-d612-4774-a436-c2352817f141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:41.935192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bac13-d612-4774-a436-c2352817f141 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:41.935504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bac13-d612-4774-a436-c2352817f141 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:41.935785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bac13-d612-4774-a436-c2352817f141 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:41.936502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bac13-d612-4774-a436-c2352817f141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:41.938130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bac13-d612-4774-a436-c2352817f141 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:41.948395 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1268/2557] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:41.952845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25257992-d435-485a-9b13-756b0b883c7d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:41.964358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25257992-d435-485a-9b13-756b0b883c7d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:41.969378 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6ac34-e768-4905-9012-02e76f86cb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:41.970279 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6ac34-e768-4905-9012-02e76f86cb41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:41.970567 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6ac34-e768-4905-9012-02e76f86cb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:41.970847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6ac34-e768-4905-9012-02e76f86cb41 admin admin] PATH_INFO: `/v3/projects/e1b4f6e21e4a404e87c04ee5ac394ca6/users/9292de923f3b49b48503ca933e93bde4/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:41.972971 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6ac34-e768-4905-9012-02e76f86cb41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b4f6e21e4a404e87c04ee5ac394ca6, user_id=9292de923f3b49b48503ca933e93bde4, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:41.975688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6ac34-e768-4905-9012-02e76f86cb41 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:41.979095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:41.991080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1290/2558] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:03:41 2022] PUT /identity/v3/projects/e1b4f6e21e4a404e87c04ee5ac394ca6/users/9292de923f3b49b48503ca933e93bde4/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:42.258097 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13346040-58e2-464e-a7b8-7903f080f3c3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:42.258614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13346040-58e2-464e-a7b8-7903f080f3c3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:42.258713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13346040-58e2-464e-a7b8-7903f080f3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:42.277847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-13346040-58e2-464e-a7b8-7903f080f3c3 None None] MFA Rules not processed for user `1157b91a306540dd98a072b1ea4941b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:42.279047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-13346040-58e2-464e-a7b8-7903f080f3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:42.332876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1269/2559] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:43.882458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2aff3-c199-4994-bce5-184803229c63 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:43.882953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2aff3-c199-4994-bce5-184803229c63 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:43.883165 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2aff3-c199-4994-bce5-184803229c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:43.913566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-70a2aff3-c199-4994-bce5-184803229c63 None None] MFA Rules not processed for user `9292de923f3b49b48503ca933e93bde4`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:43.914934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-70a2aff3-c199-4994-bce5-184803229c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:43.985672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1291/2560] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:43.996949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13346040-58e2-464e-a7b8-7903f080f3c3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:43.998371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-13346040-58e2-464e-a7b8-7903f080f3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:44.055924 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13346040-58e2-464e-a7b8-7903f080f3c3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:44.059542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabdafd-6e9b-464b-ae78-82b64384aca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:44.060190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabdafd-6e9b-464b-ae78-82b64384aca9 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:44.060477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabdafd-6e9b-464b-ae78-82b64384aca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:44.060754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabdafd-6e9b-464b-ae78-82b64384aca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:44.101639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:44.118251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabdafd-6e9b-464b-ae78-82b64384aca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:44.119797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabdafd-6e9b-464b-ae78-82b64384aca9 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:44.129032 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1270/2561] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:47.000236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2aff3-c199-4994-bce5-184803229c63 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:47.013158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2aff3-c199-4994-bce5-184803229c63 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:47.023827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223440e-3169-46e9-8944-4a4f7cedea09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:47.025251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223440e-3169-46e9-8944-4a4f7cedea09 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:47.025780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223440e-3169-46e9-8944-4a4f7cedea09 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:47.026215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223440e-3169-46e9-8944-4a4f7cedea09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:47.049103 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223440e-3169-46e9-8944-4a4f7cedea09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:47.050589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223440e-3169-46e9-8944-4a4f7cedea09 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:47.085893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1292/2562] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:47.840340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabdafd-6e9b-464b-ae78-82b64384aca9 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:47.909045 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabdafd-6e9b-464b-ae78-82b64384aca9 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:47.912722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:47.913552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:47.914011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:47.914440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:47.916986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:47.927585 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:47.929281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:47.939421 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1271/2563] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:53.255629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223440e-3169-46e9-8944-4a4f7cedea09 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:53.266439 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223440e-3169-46e9-8944-4a4f7cedea09 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:53.270483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:53.271142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:53.271648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:53.272089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:53.274421 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:53.346310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:53.347824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:53.357849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1293/2564] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:03:57.589098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:57.590555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:57.605357 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84e21b-0343-458b-873c-bbb76894fff5 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:57.610306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:57.611418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:57.611762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:57.612046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] PATH_INFO: `/v3/users/ac8dfa5c97594e74a6839d91b292adcd` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:57.626764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8dfa5c97594e74a6839d91b292adcd)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:57.629046 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:57.809477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8dfa5c97594e74a6839d91b292adcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:57.823110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8dfa5c97594e74a6839d91b292adcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:57.824217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8dfa5c97594e74a6839d91b292adcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:57.824545 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8dfa5c97594e74a6839d91b292adcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:57.832295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e0f8f510-b02e-41ab-93b1-7231749994e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8dfa5c97594e74a6839d91b292adcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:57.953937 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1272/2565] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:03:57 2022] DELETE /identity/v3/users/ac8dfa5c97594e74a6839d91b292adcd => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:58.255547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:58.264706 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08b474-64db-43f2-8c26-c76416633302 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:58.332051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:58.332904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:58.333195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:58.333476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] PATH_INFO: `/v3/projects/7df1ad577ddf4dc8a9a75ad1a1337ffe` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:58.336218 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:58.345090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df1ad577ddf4dc8a9a75ad1a1337ffe)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:58.347198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:58.505809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df1ad577ddf4dc8a9a75ad1a1337ffe', 'request_id': 'req-35da6a11-08d4-473a-9b33-ddfdb562d70e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:58.511744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df1ad577ddf4dc8a9a75ad1a1337ffe', 'request_id': 'req-35da6a11-08d4-473a-9b33-ddfdb562d70e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:58.596483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df1ad577ddf4dc8a9a75ad1a1337ffe', 'request_id': 'req-35da6a11-08d4-473a-9b33-ddfdb562d70e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:58.609390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] The token cache is being invalidate because project 7df1ad577ddf4dc8a9a75ad1a1337ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:03:58.609681 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:58.610317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-35da6a11-08d4-473a-9b33-ddfdb562d70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:03:58.611885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1294/2566] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:03:58 2022] DELETE /identity/v3/projects/7df1ad577ddf4dc8a9a75ad1a1337ffe => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:58.616959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dd079-864e-4ee5-bd09-16d379276bdd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:58.617467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dd079-864e-4ee5-bd09-16d379276bdd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:58.617569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dd079-864e-4ee5-bd09-16d379276bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:58.634834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f64dd079-864e-4ee5-bd09-16d379276bdd None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:58.636083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f64dd079-864e-4ee5-bd09-16d379276bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:58.679083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:58.696540 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1273/2567] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:03:58.704340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fb8b-795b-44f3-8411-422ef61b011b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:58.704631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fb8b-795b-44f3-8411-422ef61b011b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:58.704915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fb8b-795b-44f3-8411-422ef61b011b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:58.726151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-97f3fb8b-795b-44f3-8411-422ef61b011b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:58.727550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-97f3fb8b-795b-44f3-8411-422ef61b011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:58.737137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1295/2568] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:58.741530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dd079-864e-4ee5-bd09-16d379276bdd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:58.751183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dd079-864e-4ee5-bd09-16d379276bdd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:58.755810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:58.756406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:58.756691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:58.756967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:58.757689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:03:58.758021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:58.759770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:58.775918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1274/2569] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:03:58.777280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3fb8b-795b-44f3-8411-422ef61b011b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:58.787596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3fb8b-795b-44f3-8411-422ef61b011b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:58.793096 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242792-549c-4b0c-befc-2d527c82e2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:58.793766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242792-549c-4b0c-befc-2d527c82e2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:58.794047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242792-549c-4b0c-befc-2d527c82e2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:58.794335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95242792-549c-4b0c-befc-2d527c82e2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:58.795009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95242792-549c-4b0c-befc-2d527c82e2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:58.796955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95242792-549c-4b0c-befc-2d527c82e2f6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:58.918737 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1296/2570] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:03:58 2022] POST /identity/v3/projects => generated 405 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:03:58.920003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:58.930052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50df087-74a7-45fd-ab4c-0d4e95830b10 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:58.988257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ce8b6-0d8a-4beb-8e55-1b8b317716ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:58.988964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ce8b6-0d8a-4beb-8e55-1b8b317716ae admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:58.989278 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ce8b6-0d8a-4beb-8e55-1b8b317716ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:58.989559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ce8b6-0d8a-4beb-8e55-1b8b317716ae admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:58.990288 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ce8b6-0d8a-4beb-8e55-1b8b317716ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:58.992340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ce8b6-0d8a-4beb-8e55-1b8b317716ae admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:59.173664 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1275/2571] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:03:58 2022] POST /identity/v3/users => generated 303 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:03:59.174881 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242792-549c-4b0c-befc-2d527c82e2f6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:59.185686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95242792-549c-4b0c-befc-2d527c82e2f6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:59.190237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9b46a-d4c1-4ca1-87d6-108f716f7a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:59.190907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9b46a-d4c1-4ca1-87d6-108f716f7a70 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:59.191197 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9b46a-d4c1-4ca1-87d6-108f716f7a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:59.191524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9b46a-d4c1-4ca1-87d6-108f716f7a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:59.192248 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9b46a-d4c1-4ca1-87d6-108f716f7a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:59.193859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9b46a-d4c1-4ca1-87d6-108f716f7a70 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:59.207779 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1297/2572] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:59.208954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ce8b6-0d8a-4beb-8e55-1b8b317716ae admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:59.219572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ce8b6-0d8a-4beb-8e55-1b8b317716ae admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:59.224769 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108d691-5290-490d-bc42-535c9a475e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:59.225490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108d691-5290-490d-bc42-535c9a475e12 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:59.225743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108d691-5290-490d-bc42-535c9a475e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:59.226026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108d691-5290-490d-bc42-535c9a475e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:59.226755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108d691-5290-490d-bc42-535c9a475e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:59.228716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108d691-5290-490d-bc42-535c9a475e12 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:59.240817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1276/2573] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:59.250033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9b46a-d4c1-4ca1-87d6-108f716f7a70 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:59.261812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9b46a-d4c1-4ca1-87d6-108f716f7a70 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:59.266670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0424-438f-492b-b2f3-a5f8a989dce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:59.267616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0424-438f-492b-b2f3-a5f8a989dce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:59.267914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0424-438f-492b-b2f3-a5f8a989dce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:59.268189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0424-438f-492b-b2f3-a5f8a989dce4 admin admin] PATH_INFO: `/v3/projects/fd8489de86054bc0a6470e4e2659d464/users/1771825c27bd4ddda9d1406274409403/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:59.286444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab0424-438f-492b-b2f3-a5f8a989dce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8489de86054bc0a6470e4e2659d464, user_id=1771825c27bd4ddda9d1406274409403, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:59.289429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab0424-438f-492b-b2f3-a5f8a989dce4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:59.443818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1298/2574] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:03:59 2022] PUT /identity/v3/projects/fd8489de86054bc0a6470e4e2659d464/users/1771825c27bd4ddda9d1406274409403/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:03:59.451837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292f4a-ccb2-4b45-9f92-7b7f5740cde9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:59.452138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292f4a-ccb2-4b45-9f92-7b7f5740cde9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:59.452446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e292f4a-ccb2-4b45-9f92-7b7f5740cde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:59.479809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6e292f4a-ccb2-4b45-9f92-7b7f5740cde9 None None] MFA Rules not processed for user `1771825c27bd4ddda9d1406274409403`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:03:59.481104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6e292f4a-ccb2-4b45-9f92-7b7f5740cde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:03:59.547328 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1277/2575] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:03:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:03:59.552079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0424-438f-492b-b2f3-a5f8a989dce4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:59.644074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0424-438f-492b-b2f3-a5f8a989dce4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:59.649463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac62bd1-7a70-45ce-8fce-bcc3678a7cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:59.650001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac62bd1-7a70-45ce-8fce-bcc3678a7cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:59.650291 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac62bd1-7a70-45ce-8fce-bcc3678a7cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:59.650572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac62bd1-7a70-45ce-8fce-bcc3678a7cd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:59.651311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac62bd1-7a70-45ce-8fce-bcc3678a7cd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:59.657709 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac62bd1-7a70-45ce-8fce-bcc3678a7cd3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:59.761247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1299/2576] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:03:59 2022] POST /identity/v3/projects => generated 403 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:03:59.765240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e292f4a-ccb2-4b45-9f92-7b7f5740cde9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:59.776544 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e292f4a-ccb2-4b45-9f92-7b7f5740cde9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:59.825949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:59.846883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f18f4-40ba-44de-b93f-133047fff4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:59.847633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f18f4-40ba-44de-b93f-133047fff4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:59.847980 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f18f4-40ba-44de-b93f-133047fff4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:59.848305 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f18f4-40ba-44de-b93f-133047fff4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:59.849058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f18f4-40ba-44de-b93f-133047fff4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:59.851022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f18f4-40ba-44de-b93f-133047fff4f4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:59.936708 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1278/2577] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:03:59 2022] POST /identity/v3/users => generated 308 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:03:59.940574 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac62bd1-7a70-45ce-8fce-bcc3678a7cd3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:03:59.947612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:03:59.953131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac62bd1-7a70-45ce-8fce-bcc3678a7cd3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:03:59.960162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4cfc6-9070-4e83-80e3-191cbf725fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:03:59.960927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4cfc6-9070-4e83-80e3-191cbf725fcf admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:03:59.961821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4cfc6-9070-4e83-80e3-191cbf725fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:03:59.961821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4cfc6-9070-4e83-80e3-191cbf725fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:03:59.962824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4cfc6-9070-4e83-80e3-191cbf725fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:03:59.964771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4cfc6-9070-4e83-80e3-191cbf725fcf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:03:59.986591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1300/2578] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:03:59.996175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f18f4-40ba-44de-b93f-133047fff4f4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:00.012037 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f18f4-40ba-44de-b93f-133047fff4f4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:00.018733 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed22330-568f-4a74-a678-89f43e0c12cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:00.019674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed22330-568f-4a74-a678-89f43e0c12cf admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:00.020020 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed22330-568f-4a74-a678-89f43e0c12cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:00.020245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed22330-568f-4a74-a678-89f43e0c12cf admin admin] PATH_INFO: `/v3/projects/8a0236c36b774312a277e82894d807c0/users/28d9954014904741bd4875799532c9ba/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:00.045448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed22330-568f-4a74-a678-89f43e0c12cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0236c36b774312a277e82894d807c0, user_id=28d9954014904741bd4875799532c9ba, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:00.048487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed22330-568f-4a74-a678-89f43e0c12cf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:00.188855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1279/2579] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:03:59 2022] PUT /identity/v3/projects/8a0236c36b774312a277e82894d807c0/users/28d9954014904741bd4875799532c9ba/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:00.189721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4cfc6-9070-4e83-80e3-191cbf725fcf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:00.205185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4cfc6-9070-4e83-80e3-191cbf725fcf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:00.296030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2365a01-8679-4e52-bee2-e9977b34ee1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:00.296754 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2365a01-8679-4e52-bee2-e9977b34ee1e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:00.297080 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2365a01-8679-4e52-bee2-e9977b34ee1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:00.297654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2365a01-8679-4e52-bee2-e9977b34ee1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:00.299335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2365a01-8679-4e52-bee2-e9977b34ee1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:00.300722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2365a01-8679-4e52-bee2-e9977b34ee1e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:00.313974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1301/2580] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:00.350837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed22330-568f-4a74-a678-89f43e0c12cf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:00.360690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed22330-568f-4a74-a678-89f43e0c12cf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:00.365488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:00.366448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:00.366801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:00.367459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] PATH_INFO: `/v3/projects/8a0236c36b774312a277e82894d807c0/users/28d9954014904741bd4875799532c9ba/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:00.369874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0236c36b774312a277e82894d807c0, user_id=28d9954014904741bd4875799532c9ba, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:00.372686 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:00.395499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1280/2581] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:00 2022] PUT /identity/v3/projects/8a0236c36b774312a277e82894d807c0/users/28d9954014904741bd4875799532c9ba/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:00.401239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584e9cf-cc53-4183-9de4-055de89c8cb3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:00.401756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584e9cf-cc53-4183-9de4-055de89c8cb3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:00.402490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584e9cf-cc53-4183-9de4-055de89c8cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:00.432403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a584e9cf-cc53-4183-9de4-055de89c8cb3 None None] MFA Rules not processed for user `28d9954014904741bd4875799532c9ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:00.434124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a584e9cf-cc53-4183-9de4-055de89c8cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:00.521006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1302/2582] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:00.532982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:00.534889 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:00.607397 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13932560-3a71-43c7-b748-d36638594807 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:00.612202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed7bad-8c66-4897-b98e-8ff1de72630a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:00.613390 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed7bad-8c66-4897-b98e-8ff1de72630a service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:00.614086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed7bad-8c66-4897-b98e-8ff1de72630a service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:00.614615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed7bad-8c66-4897-b98e-8ff1de72630a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:00.632430 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed7bad-8c66-4897-b98e-8ff1de72630a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:00.636120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed7bad-8c66-4897-b98e-8ff1de72630a service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:00.648443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1281/2583] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:00 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:00.828667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584e9cf-cc53-4183-9de4-055de89c8cb3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:00.893257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584e9cf-cc53-4183-9de4-055de89c8cb3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:00.897901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:00.898983 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:00.899277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:00.899589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] PATH_INFO: `/v3/users/1771825c27bd4ddda9d1406274409403` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:00.900998 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1771825c27bd4ddda9d1406274409403)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:00.903057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:01.113482 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1771825c27bd4ddda9d1406274409403'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:01.126030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:01.131817 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1771825c27bd4ddda9d1406274409403'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:01.132644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1771825c27bd4ddda9d1406274409403'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:01.133332 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1771825c27bd4ddda9d1406274409403'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:01.142506 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1771825c27bd4ddda9d1406274409403'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:01.263216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1303/2584] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:00 2022] DELETE /identity/v3/users/1771825c27bd4ddda9d1406274409403 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:01.270516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f600ca7-553a-4b82-a18f-a63cebb308bb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:01.271080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f600ca7-553a-4b82-a18f-a63cebb308bb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:01.271080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f600ca7-553a-4b82-a18f-a63cebb308bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:01.289794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9f600ca7-553a-4b82-a18f-a63cebb308bb None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:01.291086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9f600ca7-553a-4b82-a18f-a63cebb308bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:01.359526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1282/2585] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:01.367276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:01.368916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:01.434149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a88dc-b06a-44f3-8637-5fe2e7770f94 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:01.437785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f33f6-5f45-4219-b490-1cf66cfcafbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:01.438418 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f33f6-5f45-4219-b490-1cf66cfcafbc service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:01.438699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f33f6-5f45-4219-b490-1cf66cfcafbc service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:01.438976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f33f6-5f45-4219-b490-1cf66cfcafbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:01.449700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f33f6-5f45-4219-b490-1cf66cfcafbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:01.451167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f33f6-5f45-4219-b490-1cf66cfcafbc service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:01.460912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1304/2586] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:01.923685 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f600ca7-553a-4b82-a18f-a63cebb308bb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:01.925123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9f600ca7-553a-4b82-a18f-a63cebb308bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:01.937875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f600ca7-553a-4b82-a18f-a63cebb308bb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:01.942804 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:01.943713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:01.944500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:01.944809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] PATH_INFO: `/v3/projects/fd8489de86054bc0a6470e4e2659d464` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:01.946268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8489de86054bc0a6470e4e2659d464)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:01.948255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:02.070028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:02.083320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8489de86054bc0a6470e4e2659d464', 'request_id': 'req-e555a59b-e9a4-4bfe-859f-8e06d563f693'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.089737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8489de86054bc0a6470e4e2659d464', 'request_id': 'req-e555a59b-e9a4-4bfe-859f-8e06d563f693'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.095204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8489de86054bc0a6470e4e2659d464', 'request_id': 'req-e555a59b-e9a4-4bfe-859f-8e06d563f693'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.229508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] The token cache is being invalidate because project fd8489de86054bc0a6470e4e2659d464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:02.229809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.230445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.232207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1283/2587] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:01 2022] DELETE /identity/v3/projects/fd8489de86054bc0a6470e4e2659d464 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:02.236565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f33f6-5f45-4219-b490-1cf66cfcafbc service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:02.247586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f33f6-5f45-4219-b490-1cf66cfcafbc service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:02.303362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:02.304187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:02.304469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:02.304742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] PATH_INFO: `/v3/users/28d9954014904741bd4875799532c9ba` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:02.305960 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d9954014904741bd4875799532c9ba)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:02.307927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:02.466698 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:02.472715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d9954014904741bd4875799532c9ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.484131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d9954014904741bd4875799532c9ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.487723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d9954014904741bd4875799532c9ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.487723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d9954014904741bd4875799532c9ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.492019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1050bed4-e229-414d-9766-7bf2595c61b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d9954014904741bd4875799532c9ba'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:02.566977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1305/2588] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:02 2022] DELETE /identity/v3/users/28d9954014904741bd4875799532c9ba => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:02.968797 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:02.970843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:02.984490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555a59b-e9a4-4bfe-859f-8e06d563f693 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:03.044906 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:03.045764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.046050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.046326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] PATH_INFO: `/v3/projects/8a0236c36b774312a277e82894d807c0` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.047547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a0236c36b774312a277e82894d807c0)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:03.049475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:03.159373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a0236c36b774312a277e82894d807c0', 'request_id': 'req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:03.166017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a0236c36b774312a277e82894d807c0', 'request_id': 'req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:03.171645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a0236c36b774312a277e82894d807c0', 'request_id': 'req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:03.237743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] The token cache is being invalidate because project 8a0236c36b774312a277e82894d807c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:03.238041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:03.238671 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-af1d4105-85b0-4a7e-b5d2-71c3c811fd37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:03.240262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1284/2589] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:02 2022] DELETE /identity/v3/projects/8a0236c36b774312a277e82894d807c0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:03.245453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8804-9cd1-4da5-9568-a9e332bed0b6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.245744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8804-9cd1-4da5-9568-a9e332bed0b6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.246065 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad8804-9cd1-4da5-9568-a9e332bed0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.264867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-63ad8804-9cd1-4da5-9568-a9e332bed0b6 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:03.266204 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-63ad8804-9cd1-4da5-9568-a9e332bed0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:03.327414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1306/2590] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:03.339477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b5e9b-f2cf-493b-890c-759b112bc8a1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.339778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b5e9b-f2cf-493b-890c-759b112bc8a1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.340064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b5e9b-f2cf-493b-890c-759b112bc8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.358761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-314b5e9b-f2cf-493b-890c-759b112bc8a1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:03.360060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-314b5e9b-f2cf-493b-890c-759b112bc8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:03.371874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1285/2591] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:03.376159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad8804-9cd1-4da5-9568-a9e332bed0b6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:03.387258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad8804-9cd1-4da5-9568-a9e332bed0b6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:03.392410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:03.396827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.396827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.396827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.396827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:03.396827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:03.398026 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:03.412148 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1307/2592] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:04:03.416458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b5e9b-f2cf-493b-890c-759b112bc8a1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:03.425682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b5e9b-f2cf-493b-890c-759b112bc8a1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:03.430281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960da09-7e53-453f-a9ee-f09247b63bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:03.430955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960da09-7e53-453f-a9ee-f09247b63bbf admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.431239 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960da09-7e53-453f-a9ee-f09247b63bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.431548 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960da09-7e53-453f-a9ee-f09247b63bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.432234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960da09-7e53-453f-a9ee-f09247b63bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:03.434130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0960da09-7e53-453f-a9ee-f09247b63bbf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:03.548673 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1286/2593] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:03 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:03.552602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:03.562671 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47064e-a6a8-458f-8c11-07ef3c62c2d8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:03.617627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccc4a8-4530-4cc0-adc8-b436601ba339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:03.618313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccc4a8-4530-4cc0-adc8-b436601ba339 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.618596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccc4a8-4530-4cc0-adc8-b436601ba339 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.618873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccc4a8-4530-4cc0-adc8-b436601ba339 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.619584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccc4a8-4530-4cc0-adc8-b436601ba339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:03.621505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccc4a8-4530-4cc0-adc8-b436601ba339 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:03.772134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1308/2594] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:03 2022] POST /identity/v3/users => generated 284 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:03.776301 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960da09-7e53-453f-a9ee-f09247b63bbf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:03.786423 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0960da09-7e53-453f-a9ee-f09247b63bbf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:03.791044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4faf5-ace7-4654-8e51-7b2f3ed6a5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:03.791734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4faf5-ace7-4654-8e51-7b2f3ed6a5aa admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.792017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4faf5-ace7-4654-8e51-7b2f3ed6a5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.792290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4faf5-ace7-4654-8e51-7b2f3ed6a5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.792999 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4faf5-ace7-4654-8e51-7b2f3ed6a5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:03.794828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4faf5-ace7-4654-8e51-7b2f3ed6a5aa admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:03.805927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1287/2595] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:03.811785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccc4a8-4530-4cc0-adc8-b436601ba339 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:03.821719 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccc4a8-4530-4cc0-adc8-b436601ba339 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:03.827350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0158d10-e2aa-4062-9b0e-08a7c58aee43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:03.828020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158d10-e2aa-4062-9b0e-08a7c58aee43 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.828322 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158d10-e2aa-4062-9b0e-08a7c58aee43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.828628 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158d10-e2aa-4062-9b0e-08a7c58aee43 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.829386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0158d10-e2aa-4062-9b0e-08a7c58aee43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:03.830978 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0158d10-e2aa-4062-9b0e-08a7c58aee43 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:03.846561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1309/2596] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:03.850762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4faf5-ace7-4654-8e51-7b2f3ed6a5aa admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:03.861527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4faf5-ace7-4654-8e51-7b2f3ed6a5aa admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:03.866146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2febc-67c8-41f1-b54a-db2b6b3a69d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:03.867026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2febc-67c8-41f1-b54a-db2b6b3a69d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.867336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2febc-67c8-41f1-b54a-db2b6b3a69d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.867624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2febc-67c8-41f1-b54a-db2b6b3a69d8 admin admin] PATH_INFO: `/v3/projects/4baa2b9a1b42498784efe5f80961a5e3/users/3a2098ea86eb4e4e90122ed0e7717a40/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.870914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:03.882594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2febc-67c8-41f1-b54a-db2b6b3a69d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4baa2b9a1b42498784efe5f80961a5e3, user_id=3a2098ea86eb4e4e90122ed0e7717a40, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:03.885446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2febc-67c8-41f1-b54a-db2b6b3a69d8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:03.958363 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1288/2597] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:03 2022] PUT /identity/v3/projects/4baa2b9a1b42498784efe5f80961a5e3/users/3a2098ea86eb4e4e90122ed0e7717a40/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:03.970699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aee812-ed73-41e3-81a4-665f4ae15cf0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:03.970995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aee812-ed73-41e3-81a4-665f4ae15cf0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:03.971280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aee812-ed73-41e3-81a4-665f4ae15cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:03.998489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-32aee812-ed73-41e3-81a4-665f4ae15cf0 None None] MFA Rules not processed for user `3a2098ea86eb4e4e90122ed0e7717a40`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:03.999759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-32aee812-ed73-41e3-81a4-665f4ae15cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:04.014266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:04.063930 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1310/2598] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:04.068372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2febc-67c8-41f1-b54a-db2b6b3a69d8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:04.130417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2febc-67c8-41f1-b54a-db2b6b3a69d8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:04.135633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34105ed-fd81-4acc-b3c9-a071977939fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:04.136290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34105ed-fd81-4acc-b3c9-a071977939fa admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:04.136583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34105ed-fd81-4acc-b3c9-a071977939fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:04.136865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34105ed-fd81-4acc-b3c9-a071977939fa admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:04.137553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34105ed-fd81-4acc-b3c9-a071977939fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:04.139499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34105ed-fd81-4acc-b3c9-a071977939fa admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:04.214500 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1289/2599] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:04 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:04.218635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aee812-ed73-41e3-81a4-665f4ae15cf0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:04.229663 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aee812-ed73-41e3-81a4-665f4ae15cf0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:04.286741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91ae22-1fd3-4007-9d7a-7d399ebfbef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:04.287343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91ae22-1fd3-4007-9d7a-7d399ebfbef4 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:04.287631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91ae22-1fd3-4007-9d7a-7d399ebfbef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:04.287907 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91ae22-1fd3-4007-9d7a-7d399ebfbef4 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:04.288587 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91ae22-1fd3-4007-9d7a-7d399ebfbef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:04.290485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91ae22-1fd3-4007-9d7a-7d399ebfbef4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:04.373195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1311/2600] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:04 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:04.377207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34105ed-fd81-4acc-b3c9-a071977939fa admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:04.388502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34105ed-fd81-4acc-b3c9-a071977939fa admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:04.393574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6932307-4f8e-4f9c-b0b2-431414ae2e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:04.395595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6932307-4f8e-4f9c-b0b2-431414ae2e51 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:04.395595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6932307-4f8e-4f9c-b0b2-431414ae2e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:04.395595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6932307-4f8e-4f9c-b0b2-431414ae2e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:04.395595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6932307-4f8e-4f9c-b0b2-431414ae2e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:04.397178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6932307-4f8e-4f9c-b0b2-431414ae2e51 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:04.409814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1290/2601] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:04.413949 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91ae22-1fd3-4007-9d7a-7d399ebfbef4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:04.424151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91ae22-1fd3-4007-9d7a-7d399ebfbef4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:04.428771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9650dd-ca44-46f0-a354-bd28466d57eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:04.429660 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9650dd-ca44-46f0-a354-bd28466d57eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:04.429951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9650dd-ca44-46f0-a354-bd28466d57eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:04.430235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9650dd-ca44-46f0-a354-bd28466d57eb admin admin] PATH_INFO: `/v3/projects/ac65c08ee088427888c4ae5e56f1145c/users/7759d394d74d413fa8ed4400bf55fc57/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:04.445180 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9650dd-ca44-46f0-a354-bd28466d57eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac65c08ee088427888c4ae5e56f1145c, user_id=7759d394d74d413fa8ed4400bf55fc57, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:04.448496 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9650dd-ca44-46f0-a354-bd28466d57eb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:04.526863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1312/2602] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:04 2022] PUT /identity/v3/projects/ac65c08ee088427888c4ae5e56f1145c/users/7759d394d74d413fa8ed4400bf55fc57/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:04.530793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6932307-4f8e-4f9c-b0b2-431414ae2e51 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:04.540888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6932307-4f8e-4f9c-b0b2-431414ae2e51 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:04.597984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2baef-2534-456f-a73b-d02a4fbcab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:04.598645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2baef-2534-456f-a73b-d02a4fbcab8a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:04.598931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2baef-2534-456f-a73b-d02a4fbcab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:04.599211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2baef-2534-456f-a73b-d02a4fbcab8a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:04.599948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2baef-2534-456f-a73b-d02a4fbcab8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:04.601565 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2baef-2534-456f-a73b-d02a4fbcab8a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:04.611082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1291/2603] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:04.615351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9650dd-ca44-46f0-a354-bd28466d57eb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:04.626258 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9650dd-ca44-46f0-a354-bd28466d57eb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:04.630844 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:04.631761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:04.632041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:04.632318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] PATH_INFO: `/v3/projects/ac65c08ee088427888c4ae5e56f1145c/users/7759d394d74d413fa8ed4400bf55fc57/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:04.634280 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac65c08ee088427888c4ae5e56f1145c, user_id=7759d394d74d413fa8ed4400bf55fc57, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:04.637536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:04.647561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1313/2604] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:04 2022] PUT /identity/v3/projects/ac65c08ee088427888c4ae5e56f1145c/users/7759d394d74d413fa8ed4400bf55fc57/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:04.657411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cabe30-e843-40d4-a06b-5513e9406385 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:04.657703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cabe30-e843-40d4-a06b-5513e9406385 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:04.657981 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cabe30-e843-40d4-a06b-5513e9406385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:04.686648 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-41cabe30-e843-40d4-a06b-5513e9406385 None None] MFA Rules not processed for user `7759d394d74d413fa8ed4400bf55fc57`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:04.687900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-41cabe30-e843-40d4-a06b-5513e9406385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:04.740773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:04.765826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1292/2605] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:04.777284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:04.778962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:04.846056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b286ad-85c8-4860-b40f-3c0f18d3d534 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:04.850041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082f2dd-1cb9-4f73-b20e-ea9d35f28979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:04.850775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082f2dd-1cb9-4f73-b20e-ea9d35f28979 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:04.851001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082f2dd-1cb9-4f73-b20e-ea9d35f28979 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:04.851278 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082f2dd-1cb9-4f73-b20e-ea9d35f28979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:04.872948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082f2dd-1cb9-4f73-b20e-ea9d35f28979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:04.872948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082f2dd-1cb9-4f73-b20e-ea9d35f28979 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:04.877509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1314/2606] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:07.989552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cabe30-e843-40d4-a06b-5513e9406385 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:08.055460 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cabe30-e843-40d4-a06b-5513e9406385 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:08.060581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:08.065230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:08.065230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:08.065230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] PATH_INFO: `/v3/users/3a2098ea86eb4e4e90122ed0e7717a40` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:08.065230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2098ea86eb4e4e90122ed0e7717a40)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:08.065230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:08.293542 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2098ea86eb4e4e90122ed0e7717a40'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:08.305300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2098ea86eb4e4e90122ed0e7717a40'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:08.305995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2098ea86eb4e4e90122ed0e7717a40'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:08.306612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2098ea86eb4e4e90122ed0e7717a40'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:08.313501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2098ea86eb4e4e90122ed0e7717a40'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:08.433233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1293/2607] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:07 2022] DELETE /identity/v3/users/3a2098ea86eb4e4e90122ed0e7717a40 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:08.437748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0b3ef-271f-4a73-83d0-4fdf35ce371b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:08.438231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0b3ef-271f-4a73-83d0-4fdf35ce371b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:08.438343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0b3ef-271f-4a73-83d0-4fdf35ce371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:08.450209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:08.456192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ccd0b3ef-271f-4a73-83d0-4fdf35ce371b None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:08.457445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ccd0b3ef-271f-4a73-83d0-4fdf35ce371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:08.516955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1315/2608] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:08.527051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:08.528507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:08.587855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8270c-48c5-4701-b580-4ba027fdcb17 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:08.592555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9624ed-443c-4196-88ea-6e935332f853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:08.593204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9624ed-443c-4196-88ea-6e935332f853 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:08.593493 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9624ed-443c-4196-88ea-6e935332f853 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:08.593775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9624ed-443c-4196-88ea-6e935332f853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:08.603253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9624ed-443c-4196-88ea-6e935332f853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:08.604736 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9624ed-443c-4196-88ea-6e935332f853 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:08.614412 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1294/2609] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:09.106149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0b3ef-271f-4a73-83d0-4fdf35ce371b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:09.107629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ccd0b3ef-271f-4a73-83d0-4fdf35ce371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:09.122032 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0b3ef-271f-4a73-83d0-4fdf35ce371b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:09.127007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:09.128610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:09.128921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:09.129207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] PATH_INFO: `/v3/projects/4baa2b9a1b42498784efe5f80961a5e3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:09.130492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4baa2b9a1b42498784efe5f80961a5e3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:09.132463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:09.270692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4baa2b9a1b42498784efe5f80961a5e3', 'request_id': 'req-0a27e55e-4ba2-4870-b8cd-d99d8e409253'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.276063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4baa2b9a1b42498784efe5f80961a5e3', 'request_id': 'req-0a27e55e-4ba2-4870-b8cd-d99d8e409253'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.361283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4baa2b9a1b42498784efe5f80961a5e3', 'request_id': 'req-0a27e55e-4ba2-4870-b8cd-d99d8e409253'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.373687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] The token cache is being invalidate because project 4baa2b9a1b42498784efe5f80961a5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:09.373981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.374621 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.376235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1316/2610] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:09 2022] DELETE /identity/v3/projects/4baa2b9a1b42498784efe5f80961a5e3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:09.380371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9624ed-443c-4196-88ea-6e935332f853 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:09.386968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:09.392250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9624ed-443c-4196-88ea-6e935332f853 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:09.449220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:09.450089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:09.450379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:09.450662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] PATH_INFO: `/v3/users/7759d394d74d413fa8ed4400bf55fc57` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:09.451881 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7759d394d74d413fa8ed4400bf55fc57)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:09.453823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:09.669680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7759d394d74d413fa8ed4400bf55fc57'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.681099 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7759d394d74d413fa8ed4400bf55fc57'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.682401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7759d394d74d413fa8ed4400bf55fc57'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.683007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7759d394d74d413fa8ed4400bf55fc57'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.689762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8fc32197-7171-4939-8274-bd63fec5e418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7759d394d74d413fa8ed4400bf55fc57'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:09.767631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1295/2611] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:09 2022] DELETE /identity/v3/users/7759d394d74d413fa8ed4400bf55fc57 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:10.205338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:10.206829 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:10.222146 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27e55e-4ba2-4870-b8cd-d99d8e409253 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:10.280590 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:10.281461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:10.281750 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:10.282049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] PATH_INFO: `/v3/projects/ac65c08ee088427888c4ae5e56f1145c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:10.283326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac65c08ee088427888c4ae5e56f1145c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:10.285276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:10.475833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:10.487348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac65c08ee088427888c4ae5e56f1145c', 'request_id': 'req-24b63f99-952d-4dca-bba1-2bf67e34957e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:10.492928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac65c08ee088427888c4ae5e56f1145c', 'request_id': 'req-24b63f99-952d-4dca-bba1-2bf67e34957e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:10.599576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac65c08ee088427888c4ae5e56f1145c', 'request_id': 'req-24b63f99-952d-4dca-bba1-2bf67e34957e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:10.612005 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] The token cache is being invalidate because project ac65c08ee088427888c4ae5e56f1145c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:10.612299 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:10.612959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-24b63f99-952d-4dca-bba1-2bf67e34957e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:10.614725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1317/2612] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:10 2022] DELETE /identity/v3/projects/ac65c08ee088427888c4ae5e56f1145c => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:10.621965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf75e6d-9ad7-4c70-829d-360e480c8e03 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:10.622444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf75e6d-9ad7-4c70-829d-360e480c8e03 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:10.622561 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf75e6d-9ad7-4c70-829d-360e480c8e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:10.641521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7cf75e6d-9ad7-4c70-829d-360e480c8e03 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:10.642710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7cf75e6d-9ad7-4c70-829d-360e480c8e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:10.704811 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1296/2613] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:10.712675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35756e-87ae-4d3b-964e-ed7af1c5f9ea None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:10.713009 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35756e-87ae-4d3b-964e-ed7af1c5f9ea None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:10.713304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35756e-87ae-4d3b-964e-ed7af1c5f9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:10.732814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ef35756e-87ae-4d3b-964e-ed7af1c5f9ea None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:10.733969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ef35756e-87ae-4d3b-964e-ed7af1c5f9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:10.752388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1318/2614] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:10.756659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf75e6d-9ad7-4c70-829d-360e480c8e03 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:10.767032 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf75e6d-9ad7-4c70-829d-360e480c8e03 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:10.771806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:10.772469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:10.772757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:10.773051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:10.773753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:10.774063 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:10.776038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:10.789267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1297/2615] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 16:04:10.794135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35756e-87ae-4d3b-964e-ed7af1c5f9ea None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:10.803485 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35756e-87ae-4d3b-964e-ed7af1c5f9ea None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:10.808823 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d686cad-62c1-43fe-aa27-88af7017738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:10.809470 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686cad-62c1-43fe-aa27-88af7017738b admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:10.809755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686cad-62c1-43fe-aa27-88af7017738b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:10.810037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d686cad-62c1-43fe-aa27-88af7017738b admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:10.810720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d686cad-62c1-43fe-aa27-88af7017738b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:10.812670 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d686cad-62c1-43fe-aa27-88af7017738b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:10.921593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1319/2616] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:10 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:10.929124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:10.941945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2287e2-bbc2-4653-a873-796509133c10 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:10.951260 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:11.006993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f01c8-3e6c-4dcb-8463-a038581851c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:11.007848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f01c8-3e6c-4dcb-8463-a038581851c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.008141 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f01c8-3e6c-4dcb-8463-a038581851c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.008422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f01c8-3e6c-4dcb-8463-a038581851c8 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.009115 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f01c8-3e6c-4dcb-8463-a038581851c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:11.011250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f01c8-3e6c-4dcb-8463-a038581851c8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:11.115688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1298/2617] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:10 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:11.120512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d686cad-62c1-43fe-aa27-88af7017738b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:11.130761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d686cad-62c1-43fe-aa27-88af7017738b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:11.135346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990726c0-8b78-4ec0-affb-c9b749fe572c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:11.136645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990726c0-8b78-4ec0-affb-c9b749fe572c admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.136931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990726c0-8b78-4ec0-affb-c9b749fe572c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.137212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990726c0-8b78-4ec0-affb-c9b749fe572c admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.137934 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990726c0-8b78-4ec0-affb-c9b749fe572c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:11.139578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990726c0-8b78-4ec0-affb-c9b749fe572c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:11.150184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1320/2618] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:11.154201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f01c8-3e6c-4dcb-8463-a038581851c8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:11.164508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f01c8-3e6c-4dcb-8463-a038581851c8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:11.169813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028b922-973c-4856-b276-3ae3cc5b410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:11.170464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028b922-973c-4856-b276-3ae3cc5b410c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.170755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028b922-973c-4856-b276-3ae3cc5b410c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.171038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028b922-973c-4856-b276-3ae3cc5b410c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.172578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028b922-973c-4856-b276-3ae3cc5b410c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:11.173523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028b922-973c-4856-b276-3ae3cc5b410c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:11.184901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1299/2619] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:11.189086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990726c0-8b78-4ec0-affb-c9b749fe572c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:11.199368 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990726c0-8b78-4ec0-affb-c9b749fe572c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:11.204286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8191a83-108d-4dce-8f7c-d638e96d2443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:11.205162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8191a83-108d-4dce-8f7c-d638e96d2443 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.205448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8191a83-108d-4dce-8f7c-d638e96d2443 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.206014 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8191a83-108d-4dce-8f7c-d638e96d2443 admin admin] PATH_INFO: `/v3/projects/aa22111cc2254c6c84cdd812e345b006/users/502f3eac328644af8de43d63811dc35d/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.222205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8191a83-108d-4dce-8f7c-d638e96d2443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa22111cc2254c6c84cdd812e345b006, user_id=502f3eac328644af8de43d63811dc35d, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:11.224909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8191a83-108d-4dce-8f7c-d638e96d2443 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:11.304722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1321/2620] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:11 2022] PUT /identity/v3/projects/aa22111cc2254c6c84cdd812e345b006/users/502f3eac328644af8de43d63811dc35d/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:11.316111 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad99c6-9c04-4c26-abff-5f2a1478172f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.316483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad99c6-9c04-4c26-abff-5f2a1478172f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.316778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad99c6-9c04-4c26-abff-5f2a1478172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.360531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1bad99c6-9c04-4c26-abff-5f2a1478172f None None] MFA Rules not processed for user `502f3eac328644af8de43d63811dc35d`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:11.360531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1bad99c6-9c04-4c26-abff-5f2a1478172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:11.445993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1300/2621] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:11.450752 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8191a83-108d-4dce-8f7c-d638e96d2443 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:11.516390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8191a83-108d-4dce-8f7c-d638e96d2443 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:11.521038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde5aa8-89f7-41c3-9bec-30fa5cca1089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:11.521797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde5aa8-89f7-41c3-9bec-30fa5cca1089 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.522147 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde5aa8-89f7-41c3-9bec-30fa5cca1089 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.522500 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde5aa8-89f7-41c3-9bec-30fa5cca1089 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.523111 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde5aa8-89f7-41c3-9bec-30fa5cca1089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:11.525058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde5aa8-89f7-41c3-9bec-30fa5cca1089 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:11.642970 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1322/2622] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:11 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:11.646992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad99c6-9c04-4c26-abff-5f2a1478172f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:11.658600 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad99c6-9c04-4c26-abff-5f2a1478172f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:11.714884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2ba8d-11d3-4a6d-a7d5-1e25658852a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:11.715633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2ba8d-11d3-4a6d-a7d5-1e25658852a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.715919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2ba8d-11d3-4a6d-a7d5-1e25658852a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.716200 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2ba8d-11d3-4a6d-a7d5-1e25658852a3 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.716934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2ba8d-11d3-4a6d-a7d5-1e25658852a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:11.718878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2ba8d-11d3-4a6d-a7d5-1e25658852a3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:11.830800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1301/2623] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:11 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:11.832179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde5aa8-89f7-41c3-9bec-30fa5cca1089 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:11.842771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde5aa8-89f7-41c3-9bec-30fa5cca1089 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:11.847498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419834b-762c-42ed-ae28-6857aba2a845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:11.848231 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419834b-762c-42ed-ae28-6857aba2a845 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.848514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419834b-762c-42ed-ae28-6857aba2a845 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.848792 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419834b-762c-42ed-ae28-6857aba2a845 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.849514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419834b-762c-42ed-ae28-6857aba2a845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:11.851116 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419834b-762c-42ed-ae28-6857aba2a845 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:11.862774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1323/2624] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:11.866918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2ba8d-11d3-4a6d-a7d5-1e25658852a3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:11.878522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2ba8d-11d3-4a6d-a7d5-1e25658852a3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:11.883182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60331481-0d07-4d4c-a122-fe502e572e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:11.884129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331481-0d07-4d4c-a122-fe502e572e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:11.884422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331481-0d07-4d4c-a122-fe502e572e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:11.884783 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60331481-0d07-4d4c-a122-fe502e572e7c admin admin] PATH_INFO: `/v3/projects/141a155a89e14d8394f61f6e3ff764a6/users/b710dbb9bc644a20804c9d4531006cf7/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:11.887082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:11.899911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60331481-0d07-4d4c-a122-fe502e572e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=141a155a89e14d8394f61f6e3ff764a6, user_id=b710dbb9bc644a20804c9d4531006cf7, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:11.902640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60331481-0d07-4d4c-a122-fe502e572e7c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:11.976620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1302/2625] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:11 2022] PUT /identity/v3/projects/141a155a89e14d8394f61f6e3ff764a6/users/b710dbb9bc644a20804c9d4531006cf7/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:11.981276 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419834b-762c-42ed-ae28-6857aba2a845 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:11.991505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419834b-762c-42ed-ae28-6857aba2a845 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:12.024692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:12.052249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25ee47-10ef-4abe-8834-9d29b1a0430d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:12.052918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ee47-10ef-4abe-8834-9d29b1a0430d admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:12.053250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ee47-10ef-4abe-8834-9d29b1a0430d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:12.053478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25ee47-10ef-4abe-8834-9d29b1a0430d admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:12.054189 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25ee47-10ef-4abe-8834-9d29b1a0430d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:12.055834 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25ee47-10ef-4abe-8834-9d29b1a0430d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:12.065642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1324/2626] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:12.069758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60331481-0d07-4d4c-a122-fe502e572e7c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:12.080152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60331481-0d07-4d4c-a122-fe502e572e7c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:12.084801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:12.085704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:12.086075 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:12.086361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] PATH_INFO: `/v3/projects/141a155a89e14d8394f61f6e3ff764a6/users/b710dbb9bc644a20804c9d4531006cf7/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:12.089222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141a155a89e14d8394f61f6e3ff764a6, user_id=b710dbb9bc644a20804c9d4531006cf7, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:12.091988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:12.114760 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1303/2627] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:12 2022] PUT /identity/v3/projects/141a155a89e14d8394f61f6e3ff764a6/users/b710dbb9bc644a20804c9d4531006cf7/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:12.124937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f6d41-32a7-4a03-a30b-cca3502a1c00 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:12.125235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f6d41-32a7-4a03-a30b-cca3502a1c00 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:12.125524 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f6d41-32a7-4a03-a30b-cca3502a1c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:12.152944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ae9f6d41-32a7-4a03-a30b-cca3502a1c00 None None] MFA Rules not processed for user `b710dbb9bc644a20804c9d4531006cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:12.154229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ae9f6d41-32a7-4a03-a30b-cca3502a1c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:12.222860 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1325/2628] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:12.235734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:12.237501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:12.297705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5233e7c-d9bf-4bf2-8242-d3fb7cba94d2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:12.307892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a470e4-92fb-4759-bd21-4ad6614cca3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:12.309880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a470e4-92fb-4759-bd21-4ad6614cca3d service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:12.310177 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a470e4-92fb-4759-bd21-4ad6614cca3d service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:12.310465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a470e4-92fb-4759-bd21-4ad6614cca3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:12.321427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a470e4-92fb-4759-bd21-4ad6614cca3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:12.323592 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a470e4-92fb-4759-bd21-4ad6614cca3d service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:12.333450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1304/2629] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:04:12.793835 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f6d41-32a7-4a03-a30b-cca3502a1c00 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:12.795273 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ae9f6d41-32a7-4a03-a30b-cca3502a1c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:12.861360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f6d41-32a7-4a03-a30b-cca3502a1c00 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:12.865884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:12.866764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:12.867046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:12.867347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] PATH_INFO: `/v3/users/f15e8186fd88444994c75043dd09a0f3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:12.868542 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15e8186fd88444994c75043dd09a0f3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:12.870517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:13.100847 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15e8186fd88444994c75043dd09a0f3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.106084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:13.112902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15e8186fd88444994c75043dd09a0f3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.113515 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15e8186fd88444994c75043dd09a0f3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.114183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15e8186fd88444994c75043dd09a0f3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.120992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15e8186fd88444994c75043dd09a0f3'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.241441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1326/2630] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:12 2022] DELETE /identity/v3/users/f15e8186fd88444994c75043dd09a0f3 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:13.492438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a470e4-92fb-4759-bd21-4ad6614cca3d service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:13.502192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a470e4-92fb-4759-bd21-4ad6614cca3d service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:13.535403 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:13.540556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:13.565512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b710dbb9bc644a20804c9d4531006cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '141a155a89e14d8394f61f6e3ff764a6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:13.566143 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:13.566474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:13.566723 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:13.567487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:13.567812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:13.569444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:13.585110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1305/2631] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:04:13.586104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:13.587851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:13.601039 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed362e-adc8-4717-adb7-5a78b1dfee94 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:13.606371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7e16-4a67-40cf-92e0-33456fa3fbee tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b710dbb9bc644a20804c9d4531006cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '141a155a89e14d8394f61f6e3ff764a6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:13.606620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7e16-4a67-40cf-92e0-33456fa3fbee tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:13.606914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7e16-4a67-40cf-92e0-33456fa3fbee tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:13.607201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7e16-4a67-40cf-92e0-33456fa3fbee tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:13.607341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5cc86-e24c-4fca-bbfb-681f0aa8e92c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:13.607953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc7e16-4a67-40cf-92e0-33456fa3fbee tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:13.609874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc7e16-4a67-40cf-92e0-33456fa3fbee tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:13.611923 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:13.612787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:13.613074 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:13.613352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] PATH_INFO: `/v3/projects/2aebb19f9c6444fbbe7420b90cd3709e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:13.614822 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aebb19f9c6444fbbe7420b90cd3709e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:13.616772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:13.731871 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1306/2632] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:13 2022] POST /identity/v3/projects => generated 353 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:04:13.735896 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7e16-4a67-40cf-92e0-33456fa3fbee tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:13.764926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aebb19f9c6444fbbe7420b90cd3709e', 'request_id': 'req-88d554f7-91eb-4695-8be6-3b5ac9845488'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.770717 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aebb19f9c6444fbbe7420b90cd3709e', 'request_id': 'req-88d554f7-91eb-4695-8be6-3b5ac9845488'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.781602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:13.797303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7e16-4a67-40cf-92e0-33456fa3fbee tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:13.850599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b710dbb9bc644a20804c9d4531006cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '141a155a89e14d8394f61f6e3ff764a6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:13.851261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:13.851577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:13.851858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:13.852579 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:13.852886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:13.855016 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:13.865975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1307/2633] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 16:04:13.887191 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aebb19f9c6444fbbe7420b90cd3709e', 'request_id': 'req-88d554f7-91eb-4695-8be6-3b5ac9845488'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.900933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] The token cache is being invalidate because project 2aebb19f9c6444fbbe7420b90cd3709e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:13.906607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.906728 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:13.906833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:13.906833 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1327/2634] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:13 2022] DELETE /identity/v3/projects/2aebb19f9c6444fbbe7420b90cd3709e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:13.908321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:13.909720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:13.912110 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de09a7-c895-480d-a7f4-9b16bc8d5161 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:13.917325 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b710dbb9bc644a20804c9d4531006cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '141a155a89e14d8394f61f6e3ff764a6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:13.918181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:13.918475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:13.918764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] PATH_INFO: `/v3/projects/b01deec5b71a41ddad3f4410ec34353a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:13.919969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b01deec5b71a41ddad3f4410ec34353a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:13.921893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:13.974675 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d554f7-91eb-4695-8be6-3b5ac9845488 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:13.979824 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:13.980689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:13.980973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:13.981247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] PATH_INFO: `/v3/users/9292de923f3b49b48503ca933e93bde4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:13.982483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9292de923f3b49b48503ca933e93bde4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:13.984385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:13.986578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:14.059188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01deec5b71a41ddad3f4410ec34353a', 'request_id': 'req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.065894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01deec5b71a41ddad3f4410ec34353a', 'request_id': 'req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.071285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01deec5b71a41ddad3f4410ec34353a', 'request_id': 'req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.189480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] The token cache is being invalidate because project b01deec5b71a41ddad3f4410ec34353a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:14.189781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.190427 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.192053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1308/2635] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:13 2022] DELETE /identity/v3/projects/b01deec5b71a41ddad3f4410ec34353a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:14.198369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9292de923f3b49b48503ca933e93bde4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.210369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9292de923f3b49b48503ca933e93bde4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.211061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9292de923f3b49b48503ca933e93bde4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.211762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9292de923f3b49b48503ca933e93bde4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.219315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9292de923f3b49b48503ca933e93bde4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:14.521493 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1328/2636] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:13 2022] DELETE /identity/v3/users/9292de923f3b49b48503ca933e93bde4 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:14.794442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:14.796003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:14.866264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9208e-66fe-4eb0-80e4-72a3fcc4a5a8 tempest-VolumeQuotaClassesTest-1303141752 tempest-VolumeQuotaClassesTest-1303141752-project-admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:14.877571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:14.922119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:14.922977 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:14.923255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:14.923564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] PATH_INFO: `/v3/projects/e1b4f6e21e4a404e87c04ee5ac394ca6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:14.924755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b4f6e21e4a404e87c04ee5ac394ca6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:14.926718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.059332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b4f6e21e4a404e87c04ee5ac394ca6', 'request_id': 'req-59144103-e1dd-4459-b036-0ee12dc8ede0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.065354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b4f6e21e4a404e87c04ee5ac394ca6', 'request_id': 'req-59144103-e1dd-4459-b036-0ee12dc8ede0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.070688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b4f6e21e4a404e87c04ee5ac394ca6', 'request_id': 'req-59144103-e1dd-4459-b036-0ee12dc8ede0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.106003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.107456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:15.120246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e892b-757c-45c7-8eb9-3d16d34ebc0c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:15.178422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] The token cache is being invalidate because project e1b4f6e21e4a404e87c04ee5ac394ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:15.178664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:15.178794 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.179438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-59144103-e1dd-4459-b036-0ee12dc8ede0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.179558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.179803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.180078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] PATH_INFO: `/v3/users/502f3eac328644af8de43d63811dc35d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.180996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1309/2637] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:14 2022] DELETE /identity/v3/projects/e1b4f6e21e4a404e87c04ee5ac394ca6 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:15.181268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] RBAC: Authorizing `identity:delete_user(user_id=502f3eac328644af8de43d63811dc35d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:15.183233 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.186087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762860d8-b1cd-4a31-9f58-be139906006e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.186383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762860d8-b1cd-4a31-9f58-be139906006e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.186697 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762860d8-b1cd-4a31-9f58-be139906006e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.205080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-762860d8-b1cd-4a31-9f58-be139906006e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:15.206925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-762860d8-b1cd-4a31-9f58-be139906006e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:15.215964 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1310/2638] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:15.223597 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c1362-bce0-4c7b-8b45-4a8c81e3db98 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.223893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c1362-bce0-4c7b-8b45-4a8c81e3db98 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.224178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c1362-bce0-4c7b-8b45-4a8c81e3db98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.242295 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e71c1362-bce0-4c7b-8b45-4a8c81e3db98 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:15.243514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e71c1362-bce0-4c7b-8b45-4a8c81e3db98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:15.251915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1311/2639] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:15.255921 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c1362-bce0-4c7b-8b45-4a8c81e3db98 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.265490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c1362-bce0-4c7b-8b45-4a8c81e3db98 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:15.270144 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:15.271481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.271775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.272060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.272780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:15.273121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:15.274786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.286872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1312/2640] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 16:04:15.291698 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.301233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a93b7-419d-417e-b19c-3c7bd449f62b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:15.306106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:15.306519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.306820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.307103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.307819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:15.309766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.432124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502f3eac328644af8de43d63811dc35d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.443617 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502f3eac328644af8de43d63811dc35d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.444284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502f3eac328644af8de43d63811dc35d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.444891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502f3eac328644af8de43d63811dc35d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.452445 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86c5fce1-dd9f-4b08-9d06-e4a27d3f772e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502f3eac328644af8de43d63811dc35d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:15.455993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1313/2641] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:15 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:04:15.460011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.461833 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:15.521829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:15.525951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513068c-b19c-44e8-b956-d26393635b3e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:15.530596 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134ba9f-9efd-42ee-850d-a8f5f221479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:15.531257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ba9f-9efd-42ee-850d-a8f5f221479b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.531572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ba9f-9efd-42ee-850d-a8f5f221479b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.531854 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ba9f-9efd-42ee-850d-a8f5f221479b admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.532526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134ba9f-9efd-42ee-850d-a8f5f221479b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:15.534458 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134ba9f-9efd-42ee-850d-a8f5f221479b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.537490 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:15.542195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1329/2642] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:15 2022] DELETE /identity/v3/users/502f3eac328644af8de43d63811dc35d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:15.546610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564077-a723-4bf9-8c2b-42392b2b2736 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.546908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564077-a723-4bf9-8c2b-42392b2b2736 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.547192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564077-a723-4bf9-8c2b-42392b2b2736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.568672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-55564077-a723-4bf9-8c2b-42392b2b2736 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:15.568672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-55564077-a723-4bf9-8c2b-42392b2b2736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:15.622714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1330/2643] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:15.632454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55564077-a723-4bf9-8c2b-42392b2b2736 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.633816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-55564077-a723-4bf9-8c2b-42392b2b2736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:15.640062 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1314/2644] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:15 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:04:15.643957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134ba9f-9efd-42ee-850d-a8f5f221479b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.654456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134ba9f-9efd-42ee-850d-a8f5f221479b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:15.659096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33a7e-88ce-4233-8ba5-52b7f5efdd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:15.659781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33a7e-88ce-4233-8ba5-52b7f5efdd84 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.660068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33a7e-88ce-4233-8ba5-52b7f5efdd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.660398 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33a7e-88ce-4233-8ba5-52b7f5efdd84 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.661148 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e33a7e-88ce-4233-8ba5-52b7f5efdd84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:15.662757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e33a7e-88ce-4233-8ba5-52b7f5efdd84 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.673337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1315/2645] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:04:15.677102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33a7e-88ce-4233-8ba5-52b7f5efdd84 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.687469 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33a7e-88ce-4233-8ba5-52b7f5efdd84 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:15.692206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90132c5-7edf-4dbb-877e-4153a2d53916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:15.692855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90132c5-7edf-4dbb-877e-4153a2d53916 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.693150 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90132c5-7edf-4dbb-877e-4153a2d53916 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.693455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90132c5-7edf-4dbb-877e-4153a2d53916 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.694158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90132c5-7edf-4dbb-877e-4153a2d53916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:15.695429 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55564077-a723-4bf9-8c2b-42392b2b2736 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:15.695828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90132c5-7edf-4dbb-877e-4153a2d53916 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.699883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b7e80-d2d4-4cb9-99e0-965ac1252006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:15.700585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b7e80-d2d4-4cb9-99e0-965ac1252006 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.700873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b7e80-d2d4-4cb9-99e0-965ac1252006 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.701194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b7e80-d2d4-4cb9-99e0-965ac1252006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.710434 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1316/2646] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:04:15.710578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b7e80-d2d4-4cb9-99e0-965ac1252006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:15.711471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90132c5-7edf-4dbb-877e-4153a2d53916 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.711990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b7e80-d2d4-4cb9-99e0-965ac1252006 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.721929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90132c5-7edf-4dbb-877e-4153a2d53916 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:15.724352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1331/2647] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:04:15.728479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:15.729451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.729744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.730029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] PATH_INFO: `/v3/projects/f33eb0bb7e40425393e9b13f1181ffa7/users/48d8cb122ccc41e6a04742cb5bdfd34a/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.745534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33eb0bb7e40425393e9b13f1181ffa7, user_id=48d8cb122ccc41e6a04742cb5bdfd34a, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:15.748275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:15.835798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1317/2648] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:15 2022] PUT /identity/v3/projects/f33eb0bb7e40425393e9b13f1181ffa7/users/48d8cb122ccc41e6a04742cb5bdfd34a/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:15.847221 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133813c-9f0d-4f92-9785-fae925b393d0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:15.847552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133813c-9f0d-4f92-9785-fae925b393d0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:15.847837 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133813c-9f0d-4f92-9785-fae925b393d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:15.876763 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5133813c-9f0d-4f92-9785-fae925b393d0 None None] MFA Rules not processed for user `48d8cb122ccc41e6a04742cb5bdfd34a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:15.877984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5133813c-9f0d-4f92-9785-fae925b393d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:15.942715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1332/2649] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:15.954523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:15.956283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:16.020857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae7b19-592b-4ab0-bf7c-95685a1c5650 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:16.024578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:16.025315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:16.026157 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:16.026428 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:16.035449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:16.036945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:16.046759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1318/2650] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:04:16.065478 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133813c-9f0d-4f92-9785-fae925b393d0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:16.102648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:16.130264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133813c-9f0d-4f92-9785-fae925b393d0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:16.135489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:16.136357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:16.136642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:16.136919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] PATH_INFO: `/v3/users/48d8cb122ccc41e6a04742cb5bdfd34a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:16.138130 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d8cb122ccc41e6a04742cb5bdfd34a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:16.140112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:16.189104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:16.190788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:16.206499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf3810-4641-4562-aae4-6253447e9283 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:16.211282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:16.212153 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:16.212436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:16.212705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] PATH_INFO: `/v3/projects/aa22111cc2254c6c84cdd812e345b006` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:16.215119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa22111cc2254c6c84cdd812e345b006)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:16.217210 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:16.395918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d8cb122ccc41e6a04742cb5bdfd34a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.396557 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa22111cc2254c6c84cdd812e345b006', 'request_id': 'req-c0a4347d-594f-4adc-b1ee-dec8381e3135'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.403994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa22111cc2254c6c84cdd812e345b006', 'request_id': 'req-c0a4347d-594f-4adc-b1ee-dec8381e3135'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.410710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d8cb122ccc41e6a04742cb5bdfd34a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.411089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa22111cc2254c6c84cdd812e345b006', 'request_id': 'req-c0a4347d-594f-4adc-b1ee-dec8381e3135'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.411228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d8cb122ccc41e6a04742cb5bdfd34a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.411798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d8cb122ccc41e6a04742cb5bdfd34a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.419149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5deaa504-8a36-47a0-8395-a9071ae9f87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d8cb122ccc41e6a04742cb5bdfd34a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.496646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] The token cache is being invalidate because project aa22111cc2254c6c84cdd812e345b006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:16.496646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.499624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.499624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1319/2651] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:16 2022] DELETE /identity/v3/projects/aa22111cc2254c6c84cdd812e345b006 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:16.502440 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:16.503816 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:16.539843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:16.576081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4347d-594f-4adc-b1ee-dec8381e3135 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:16.581237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:16.581918 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1333/2652] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:16 2022] DELETE /identity/v3/users/48d8cb122ccc41e6a04742cb5bdfd34a => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:16.582362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:16.582461 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:16.582706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] PATH_INFO: `/v3/users/b710dbb9bc644a20804c9d4531006cf7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:16.583967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b710dbb9bc644a20804c9d4531006cf7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:16.585967 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:16.586527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d46706-3e2c-4ca0-af86-1db0c7ff3348 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:16.586893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d46706-3e2c-4ca0-af86-1db0c7ff3348 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:16.587174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d46706-3e2c-4ca0-af86-1db0c7ff3348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:16.604104 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-17d46706-3e2c-4ca0-af86-1db0c7ff3348 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:16.605513 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-17d46706-3e2c-4ca0-af86-1db0c7ff3348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:16.672170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1334/2653] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:16.685889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d46706-3e2c-4ca0-af86-1db0c7ff3348 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:16.685889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-17d46706-3e2c-4ca0-af86-1db0c7ff3348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:16.742393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d46706-3e2c-4ca0-af86-1db0c7ff3348 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:16.746575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:16.747219 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:16.747541 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:16.747859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:16.754780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:16.756966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:16.758385 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:16.768625 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1335/2654] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:04:16.789595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b710dbb9bc644a20804c9d4531006cf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.812146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b710dbb9bc644a20804c9d4531006cf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.812901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b710dbb9bc644a20804c9d4531006cf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.813550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b710dbb9bc644a20804c9d4531006cf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.821487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b710dbb9bc644a20804c9d4531006cf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:16.922205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1320/2655] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:16 2022] DELETE /identity/v3/users/b710dbb9bc644a20804c9d4531006cf7 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:17.202980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:17.204465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:17.220268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db6426-a85d-464e-a444-0e43b26801d8 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:17.285134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:17.286022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.286312 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.286589 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] PATH_INFO: `/v3/projects/f33eb0bb7e40425393e9b13f1181ffa7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.287845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33eb0bb7e40425393e9b13f1181ffa7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:17.289802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:17.408351 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33eb0bb7e40425393e9b13f1181ffa7', 'request_id': 'req-831e61ea-434a-410e-ac15-a897488b0ce7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.413997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33eb0bb7e40425393e9b13f1181ffa7', 'request_id': 'req-831e61ea-434a-410e-ac15-a897488b0ce7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.428714 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:17.430330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:17.444269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb4698-5056-4040-9b07-3b14ac1f07ce admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:17.496061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33eb0bb7e40425393e9b13f1181ffa7', 'request_id': 'req-831e61ea-434a-410e-ac15-a897488b0ce7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.503681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:17.504556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.504845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.505127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] PATH_INFO: `/v3/projects/141a155a89e14d8394f61f6e3ff764a6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.506327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141a155a89e14d8394f61f6e3ff764a6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:17.507861 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] The token cache is being invalidate because project f33eb0bb7e40425393e9b13f1181ffa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:17.508159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.508297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:17.508800 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-831e61ea-434a-410e-ac15-a897488b0ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.510732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1336/2656] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:17 2022] DELETE /identity/v3/projects/f33eb0bb7e40425393e9b13f1181ffa7 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:17.515463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edaaea-a6b3-44c9-b2bd-c4cf8650e0c2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.515757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edaaea-a6b3-44c9-b2bd-c4cf8650e0c2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.516040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edaaea-a6b3-44c9-b2bd-c4cf8650e0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.534539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f0edaaea-a6b3-44c9-b2bd-c4cf8650e0c2 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:17.535781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f0edaaea-a6b3-44c9-b2bd-c4cf8650e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:17.545977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1337/2657] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:17.553297 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72dc5-15a6-4624-865b-2694ea97de7a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.553588 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72dc5-15a6-4624-865b-2694ea97de7a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.553876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72dc5-15a6-4624-865b-2694ea97de7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.571694 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f8d72dc5-15a6-4624-865b-2694ea97de7a None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:17.586207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f8d72dc5-15a6-4624-865b-2694ea97de7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:17.586207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1338/2658] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:17.589744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d72dc5-15a6-4624-865b-2694ea97de7a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:17.598613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d72dc5-15a6-4624-865b-2694ea97de7a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:17.603182 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:17.603890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.604174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.604453 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.605157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:17.605462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:17.607060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:17.618369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1339/2659] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 16:04:17.624003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:17.633286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec5dc-a1e3-48bd-b887-57d97ba13063 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:17.638392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82885-4f4e-43b9-a18f-ed1a36a72038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:17.638990 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82885-4f4e-43b9-a18f-ed1a36a72038 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.639272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82885-4f4e-43b9-a18f-ed1a36a72038 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.639579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff82885-4f4e-43b9-a18f-ed1a36a72038 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.640232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff82885-4f4e-43b9-a18f-ed1a36a72038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:17.642061 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff82885-4f4e-43b9-a18f-ed1a36a72038 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:17.663948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:17.665238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141a155a89e14d8394f61f6e3ff764a6', 'request_id': 'req-7770fc62-0e3c-41eb-9961-6872aee56429'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.671496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141a155a89e14d8394f61f6e3ff764a6', 'request_id': 'req-7770fc62-0e3c-41eb-9961-6872aee56429'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.677060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141a155a89e14d8394f61f6e3ff764a6', 'request_id': 'req-7770fc62-0e3c-41eb-9961-6872aee56429'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.748908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1340/2660] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:17 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:17.753077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82885-4f4e-43b9-a18f-ed1a36a72038 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:17.779872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] The token cache is being invalidate because project 141a155a89e14d8394f61f6e3ff764a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:17.780171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.780815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7770fc62-0e3c-41eb-9961-6872aee56429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:17.782380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1321/2661] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:17 2022] DELETE /identity/v3/projects/141a155a89e14d8394f61f6e3ff764a6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:17.787031 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c89f5d-a7de-4c6d-8c5e-d0090811d81c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.787356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c89f5d-a7de-4c6d-8c5e-d0090811d81c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.787647 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c89f5d-a7de-4c6d-8c5e-d0090811d81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.794531 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:17.805567 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b7c89f5d-a7de-4c6d-8c5e-d0090811d81c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:17.806779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b7c89f5d-a7de-4c6d-8c5e-d0090811d81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:17.814737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1322/2662] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:17.815120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff82885-4f4e-43b9-a18f-ed1a36a72038 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:17.820090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:17.820762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.821052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.821329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.822775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:17.824087 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88739-a524-43a9-b0e4-20b5b947f6c4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.824372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88739-a524-43a9-b0e4-20b5b947f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.824643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f88739-a524-43a9-b0e4-20b5b947f6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.824812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:17.848096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e5f88739-a524-43a9-b0e4-20b5b947f6c4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:17.848096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e5f88739-a524-43a9-b0e4-20b5b947f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:17.856114 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1323/2663] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:17.860447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f88739-a524-43a9-b0e4-20b5b947f6c4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:17.869842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f88739-a524-43a9-b0e4-20b5b947f6c4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:17.875122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:17.875798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.876093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.876379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.877138 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:17.877452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:17.879073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:17.891343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1324/2664] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 16:04:17.896608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:17.906755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb50f1-114f-492f-856b-77599dd685b8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:17.911388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3807033-5e0b-42f2-b8fe-a81ab5e731b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:17.912201 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807033-5e0b-42f2-b8fe-a81ab5e731b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:17.912489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807033-5e0b-42f2-b8fe-a81ab5e731b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:17.912766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3807033-5e0b-42f2-b8fe-a81ab5e731b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:17.913436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3807033-5e0b-42f2-b8fe-a81ab5e731b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:17.915366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3807033-5e0b-42f2-b8fe-a81ab5e731b3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:17.990228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1341/2665] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:17 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:04:17.995092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:17.996762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:18.008827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2635047-7623-4a47-8a36-e2350845c50e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.014190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1c17a-24d9-45ef-99ac-4fc5dd6e1ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.014870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1c17a-24d9-45ef-99ac-4fc5dd6e1ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.019687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1c17a-24d9-45ef-99ac-4fc5dd6e1ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.019687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1c17a-24d9-45ef-99ac-4fc5dd6e1ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.019687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1c17a-24d9-45ef-99ac-4fc5dd6e1ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.019687 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1c17a-24d9-45ef-99ac-4fc5dd6e1ab7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.032761 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1342/2666] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:04:18.033858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1c17a-24d9-45ef-99ac-4fc5dd6e1ab7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.044662 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1c17a-24d9-45ef-99ac-4fc5dd6e1ab7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.046901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1325/2667] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:17 2022] POST /identity/v3/projects => generated 397 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:04:18.051780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3807033-5e0b-42f2-b8fe-a81ab5e731b3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.103060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2907b8-bbc5-46d1-aa3d-519e5a93514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.104386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2907b8-bbc5-46d1-aa3d-519e5a93514f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.104680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2907b8-bbc5-46d1-aa3d-519e5a93514f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.104965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2907b8-bbc5-46d1-aa3d-519e5a93514f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.105757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2907b8-bbc5-46d1-aa3d-519e5a93514f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.107711 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2907b8-bbc5-46d1-aa3d-519e5a93514f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.111510 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3807033-5e0b-42f2-b8fe-a81ab5e731b3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.116711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e77d02-b72a-4270-a87a-9539dc0368ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.117492 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e77d02-b72a-4270-a87a-9539dc0368ba admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.117897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e77d02-b72a-4270-a87a-9539dc0368ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.121377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e77d02-b72a-4270-a87a-9539dc0368ba admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.121537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1343/2668] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:18.121644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e77d02-b72a-4270-a87a-9539dc0368ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.121644 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e77d02-b72a-4270-a87a-9539dc0368ba admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.122329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2907b8-bbc5-46d1-aa3d-519e5a93514f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.132505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2907b8-bbc5-46d1-aa3d-519e5a93514f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.137222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb09ebf-274f-478e-a38d-445fe763fccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.138097 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09ebf-274f-478e-a38d-445fe763fccb admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.138380 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09ebf-274f-478e-a38d-445fe763fccb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.139027 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb09ebf-274f-478e-a38d-445fe763fccb admin admin] PATH_INFO: `/v3/projects/96e59fddf57d4ec6a5ac33c7c4b3d59d/users/4bf5cc2b92b94c2abadc74b041d8d5d5/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.153631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb09ebf-274f-478e-a38d-445fe763fccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e59fddf57d4ec6a5ac33c7c4b3d59d, user_id=4bf5cc2b92b94c2abadc74b041d8d5d5, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.156364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb09ebf-274f-478e-a38d-445fe763fccb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.255366 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1326/2669] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:18 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:18.255561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1344/2670] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:18 2022] PUT /identity/v3/projects/96e59fddf57d4ec6a5ac33c7c4b3d59d/users/4bf5cc2b92b94c2abadc74b041d8d5d5/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:18.257626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e77d02-b72a-4270-a87a-9539dc0368ba admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.265764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5981f64-bb29-4c57-8caf-5f1da2f0fec8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.266179 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5981f64-bb29-4c57-8caf-5f1da2f0fec8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.266561 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5981f64-bb29-4c57-8caf-5f1da2f0fec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.267778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e77d02-b72a-4270-a87a-9539dc0368ba admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.294927 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e5981f64-bb29-4c57-8caf-5f1da2f0fec8 None None] MFA Rules not processed for user `4bf5cc2b92b94c2abadc74b041d8d5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:18.296206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e5981f64-bb29-4c57-8caf-5f1da2f0fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:18.324612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77caed4-9a1b-4696-add3-1def4b5d7e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.325289 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77caed4-9a1b-4696-add3-1def4b5d7e17 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.325577 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77caed4-9a1b-4696-add3-1def4b5d7e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.325868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77caed4-9a1b-4696-add3-1def4b5d7e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.326615 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77caed4-9a1b-4696-add3-1def4b5d7e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.328255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77caed4-9a1b-4696-add3-1def4b5d7e17 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.337959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1327/2671] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:04:18.342659 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77caed4-9a1b-4696-add3-1def4b5d7e17 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.352272 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77caed4-9a1b-4696-add3-1def4b5d7e17 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.357335 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73671ac3-d0a1-4af4-a38b-8bac45151f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.357991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671ac3-d0a1-4af4-a38b-8bac45151f97 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.358393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671ac3-d0a1-4af4-a38b-8bac45151f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.358688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73671ac3-d0a1-4af4-a38b-8bac45151f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.359505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73671ac3-d0a1-4af4-a38b-8bac45151f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.359971 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1345/2672] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:18.361459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73671ac3-d0a1-4af4-a38b-8bac45151f97 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.371815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5981f64-bb29-4c57-8caf-5f1da2f0fec8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.372315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1328/2673] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:18.376235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e5981f64-bb29-4c57-8caf-5f1da2f0fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:18.376873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73671ac3-d0a1-4af4-a38b-8bac45151f97 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.388419 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73671ac3-d0a1-4af4-a38b-8bac45151f97 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.396041 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdfde5-4c9a-43ec-bb1b-58e83a6bc8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.396936 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdfde5-4c9a-43ec-bb1b-58e83a6bc8ec admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.397220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdfde5-4c9a-43ec-bb1b-58e83a6bc8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.397501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdfde5-4c9a-43ec-bb1b-58e83a6bc8ec admin admin] PATH_INFO: `/v3/projects/27c4c6ee062b4ad68484f090bfa8c70b/users/2a8ce913e96840f79ec50b9077fddf56/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.412387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:18.415640 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdfde5-4c9a-43ec-bb1b-58e83a6bc8ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c4c6ee062b4ad68484f090bfa8c70b, user_id=2a8ce913e96840f79ec50b9077fddf56, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.418572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdfde5-4c9a-43ec-bb1b-58e83a6bc8ec admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.439234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5981f64-bb29-4c57-8caf-5f1da2f0fec8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.446327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debdd6e-2089-4891-be41-a24b8e3d0c7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.446327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debdd6e-2089-4891-be41-a24b8e3d0c7e service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.446327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debdd6e-2089-4891-be41-a24b8e3d0c7e service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.446327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debdd6e-2089-4891-be41-a24b8e3d0c7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.453067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debdd6e-2089-4891-be41-a24b8e3d0c7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.454576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debdd6e-2089-4891-be41-a24b8e3d0c7e service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.464190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1346/2674] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:04:18.483925 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debdd6e-2089-4891-be41-a24b8e3d0c7e service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.494313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debdd6e-2089-4891-be41-a24b8e3d0c7e service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.499181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.500060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.500347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.500630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] PATH_INFO: `/v3/users/4bf5cc2b92b94c2abadc74b041d8d5d5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.501831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bf5cc2b92b94c2abadc74b041d8d5d5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.504093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.536555 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:18.542347 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1329/2675] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:18 2022] PUT /identity/v3/projects/27c4c6ee062b4ad68484f090bfa8c70b/users/2a8ce913e96840f79ec50b9077fddf56/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:18.552538 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d03d57-c6fa-4105-b4be-1ee6c311c993 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.552834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d03d57-c6fa-4105-b4be-1ee6c311c993 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.553118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d03d57-c6fa-4105-b4be-1ee6c311c993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.579655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-45d03d57-c6fa-4105-b4be-1ee6c311c993 None None] MFA Rules not processed for user `2a8ce913e96840f79ec50b9077fddf56`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:18.581069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-45d03d57-c6fa-4105-b4be-1ee6c311c993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:18.643525 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1330/2676] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:18.647923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d03d57-c6fa-4105-b4be-1ee6c311c993 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.708310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d03d57-c6fa-4105-b4be-1ee6c311c993 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.712915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.713575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.714107 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.714392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.715072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.717008 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:18.747021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bf5cc2b92b94c2abadc74b041d8d5d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:18.758584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bf5cc2b92b94c2abadc74b041d8d5d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:18.759257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bf5cc2b92b94c2abadc74b041d8d5d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:18.760502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bf5cc2b92b94c2abadc74b041d8d5d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:18.767260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ddf7afb7-42a7-4d78-989a-50a2fc23ab4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bf5cc2b92b94c2abadc74b041d8d5d5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:18.857487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1331/2677] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:18 2022] POST /identity/v3/projects => generated 399 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:04:18.861156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.862823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:18.881152 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1347/2678] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:18 2022] DELETE /identity/v3/users/4bf5cc2b92b94c2abadc74b041d8d5d5 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:18.884397 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b1ba1-cc68-4091-be02-57a21017fb23 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.885433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b1ba1-cc68-4091-be02-57a21017fb23 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.885920 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b1ba1-cc68-4091-be02-57a21017fb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.905350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ea8b1ba1-cc68-4091-be02-57a21017fb23 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:18.906757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ea8b1ba1-cc68-4091-be02-57a21017fb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:18.934594 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571a972-3804-42aa-a392-4484dd4f2a79 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:18.966518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1348/2679] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:18.976246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b1ba1-cc68-4091-be02-57a21017fb23 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:18.977642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ea8b1ba1-cc68-4091-be02-57a21017fb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:18.991002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d827ec-bd56-4a15-9039-48f30c2d9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:18.991729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d827ec-bd56-4a15-9039-48f30c2d9383 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:18.992013 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d827ec-bd56-4a15-9039-48f30c2d9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:18.992291 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d827ec-bd56-4a15-9039-48f30c2d9383 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:18.993003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d827ec-bd56-4a15-9039-48f30c2d9383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:18.995581 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d827ec-bd56-4a15-9039-48f30c2d9383 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:19.017667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:19.043818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b1ba1-cc68-4091-be02-57a21017fb23 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:19.049128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81cf56-3e08-46cb-863f-9ff4e9e95f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:19.049810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81cf56-3e08-46cb-863f-9ff4e9e95f89 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.050091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81cf56-3e08-46cb-863f-9ff4e9e95f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.050366 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81cf56-3e08-46cb-863f-9ff4e9e95f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:19.061337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81cf56-3e08-46cb-863f-9ff4e9e95f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:19.062790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81cf56-3e08-46cb-863f-9ff4e9e95f89 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:19.073584 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1349/2680] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:04:19.086886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1332/2681] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:18 2022] POST /identity/v3/users => generated 306 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:04:19.088217 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81cf56-3e08-46cb-863f-9ff4e9e95f89 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:19.098201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81cf56-3e08-46cb-863f-9ff4e9e95f89 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:19.105599 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:19.105766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b2f14-2dfd-4b58-a4c5-48453b78faf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:19.106296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b2f14-2dfd-4b58-a4c5-48453b78faf6 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.106586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b2f14-2dfd-4b58-a4c5-48453b78faf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.106868 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b2f14-2dfd-4b58-a4c5-48453b78faf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:19.107705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b2f14-2dfd-4b58-a4c5-48453b78faf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:19.109272 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b2f14-2dfd-4b58-a4c5-48453b78faf6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:19.120310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1350/2682] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:19.124569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d827ec-bd56-4a15-9039-48f30c2d9383 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:19.135003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d827ec-bd56-4a15-9039-48f30c2d9383 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:19.140047 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad138-8d01-4678-8d90-dbde93926ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:19.140927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad138-8d01-4678-8d90-dbde93926ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.141342 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad138-8d01-4678-8d90-dbde93926ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.141476 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad138-8d01-4678-8d90-dbde93926ed1 admin admin] PATH_INFO: `/v3/projects/62b5c0eb27dd471ebb3ec9e12fa563fe/users/e4006c81462b4255871462c9d751bcc7/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:19.158095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad138-8d01-4678-8d90-dbde93926ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b5c0eb27dd471ebb3ec9e12fa563fe, user_id=e4006c81462b4255871462c9d751bcc7, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:19.160886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad138-8d01-4678-8d90-dbde93926ed1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:19.308001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1333/2683] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:19 2022] PUT /identity/v3/projects/62b5c0eb27dd471ebb3ec9e12fa563fe/users/e4006c81462b4255871462c9d751bcc7/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:19.312091 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b2f14-2dfd-4b58-a4c5-48453b78faf6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:19.322157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b2f14-2dfd-4b58-a4c5-48453b78faf6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:19.385598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d84a08-8d99-44c5-a2ba-3edb4d9e26a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:19.386551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d84a08-8d99-44c5-a2ba-3edb4d9e26a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.386997 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d84a08-8d99-44c5-a2ba-3edb4d9e26a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.387498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d84a08-8d99-44c5-a2ba-3edb4d9e26a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:19.388393 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d84a08-8d99-44c5-a2ba-3edb4d9e26a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:19.390163 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d84a08-8d99-44c5-a2ba-3edb4d9e26a7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:19.404324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1351/2684] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:04:19.405808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad138-8d01-4678-8d90-dbde93926ed1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:19.418705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad138-8d01-4678-8d90-dbde93926ed1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:19.425893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:19.426931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.427279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.427654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] PATH_INFO: `/v3/projects/62b5c0eb27dd471ebb3ec9e12fa563fe/users/e4006c81462b4255871462c9d751bcc7/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:19.429951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b5c0eb27dd471ebb3ec9e12fa563fe, user_id=e4006c81462b4255871462c9d751bcc7, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:19.434147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:19.445103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1334/2685] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:19 2022] PUT /identity/v3/projects/62b5c0eb27dd471ebb3ec9e12fa563fe/users/e4006c81462b4255871462c9d751bcc7/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:19.459160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fddb0-a53a-4cea-b1a9-5c74f11b869e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.459483 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fddb0-a53a-4cea-b1a9-5c74f11b869e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.459769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fddb0-a53a-4cea-b1a9-5c74f11b869e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:19.488315 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7b1fddb0-a53a-4cea-b1a9-5c74f11b869e None None] MFA Rules not processed for user `e4006c81462b4255871462c9d751bcc7`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:19.489538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7b1fddb0-a53a-4cea-b1a9-5c74f11b869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:19.557028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1352/2686] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:19 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:19.567900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:19.569622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:19.602916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fddb0-a53a-4cea-b1a9-5c74f11b869e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:19.605361 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7b1fddb0-a53a-4cea-b1a9-5c74f11b869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:19.632167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65103aab-72cd-480e-94b0-da65711dab6f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:19.636165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed33ad-e752-462b-bff2-b7eac0701b10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:19.636823 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed33ad-e752-462b-bff2-b7eac0701b10 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.637109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed33ad-e752-462b-bff2-b7eac0701b10 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.637389 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed33ad-e752-462b-bff2-b7eac0701b10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:19.647124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed33ad-e752-462b-bff2-b7eac0701b10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:19.648654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed33ad-e752-462b-bff2-b7eac0701b10 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:19.659281 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:19.659662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1335/2687] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:19 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:04:19.678863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fddb0-a53a-4cea-b1a9-5c74f11b869e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:19.683586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:19.684447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.684738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.685016 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] PATH_INFO: `/v3/projects/96e59fddf57d4ec6a5ac33c7c4b3d59d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:19.686207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e59fddf57d4ec6a5ac33c7c4b3d59d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:19.688144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:19.825414 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e59fddf57d4ec6a5ac33c7c4b3d59d', 'request_id': 'req-eef026e7-7e60-4de7-931f-4d888e16d70c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:19.831044 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e59fddf57d4ec6a5ac33c7c4b3d59d', 'request_id': 'req-eef026e7-7e60-4de7-931f-4d888e16d70c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:19.958629 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed33ad-e752-462b-bff2-b7eac0701b10 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:19.968237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed33ad-e752-462b-bff2-b7eac0701b10 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:19.970352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e59fddf57d4ec6a5ac33c7c4b3d59d', 'request_id': 'req-eef026e7-7e60-4de7-931f-4d888e16d70c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:19.982775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] The token cache is being invalidate because project 96e59fddf57d4ec6a5ac33c7c4b3d59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:19.983142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:19.983893 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-eef026e7-7e60-4de7-931f-4d888e16d70c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:19.985585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1353/2688] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:19 2022] DELETE /identity/v3/projects/96e59fddf57d4ec6a5ac33c7c4b3d59d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:19.990437 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9c70-599a-4ec4-b566-2c253460c315 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:19.990741 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9c70-599a-4ec4-b566-2c253460c315 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:19.991029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca9c70-599a-4ec4-b566-2c253460c315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.009209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7bca9c70-599a-4ec4-b566-2c253460c315 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:20.011008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7bca9c70-599a-4ec4-b566-2c253460c315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:20.022505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:20.023160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.023552 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.023839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.026386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:20.035897 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:20.073243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1354/2689] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:20.082310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31383c8c-27c3-48d4-a7ef-3b5035573428 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.082550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31383c8c-27c3-48d4-a7ef-3b5035573428 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.082820 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31383c8c-27c3-48d4-a7ef-3b5035573428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.087080 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:20.089410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:20.100691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1336/2690] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:20.104235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-31383c8c-27c3-48d4-a7ef-3b5035573428 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:20.105551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-31383c8c-27c3-48d4-a7ef-3b5035573428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:20.117733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1355/2691] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:04:20.118884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:20.129674 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079d22f-5ab7-4654-be95-f48e1bcb6096 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:20.134417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:20.135657 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.135945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.136223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.136956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:20.137271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:20.138888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:20.154852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1337/2692] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:04:20.159118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31383c8c-27c3-48d4-a7ef-3b5035573428 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:20.168615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31383c8c-27c3-48d4-a7ef-3b5035573428 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:20.173330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb74e-aab2-4124-af79-85154aee4e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:20.173995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb74e-aab2-4124-af79-85154aee4e55 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.174286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb74e-aab2-4124-af79-85154aee4e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.174568 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb74e-aab2-4124-af79-85154aee4e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.175261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb74e-aab2-4124-af79-85154aee4e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:20.177242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb74e-aab2-4124-af79-85154aee4e55 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:20.368904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1356/2693] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:20 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:20.373453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:20.383893 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc43b9-a982-4274-b4d1-d35615f07c9b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:20.445511 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87be4-f946-4961-9b4c-681393538b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:20.446184 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87be4-f946-4961-9b4c-681393538b7b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.446477 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87be4-f946-4961-9b4c-681393538b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.446756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb87be4-f946-4961-9b4c-681393538b7b admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.448181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb87be4-f946-4961-9b4c-681393538b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:20.450179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb87be4-f946-4961-9b4c-681393538b7b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:20.608512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1338/2694] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:20 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:20.612772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb74e-aab2-4124-af79-85154aee4e55 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:20.623916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb74e-aab2-4124-af79-85154aee4e55 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:20.628728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89437ecd-a9b6-4cdc-972c-3380593946f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:20.629388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89437ecd-a9b6-4cdc-972c-3380593946f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.629677 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89437ecd-a9b6-4cdc-972c-3380593946f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.629958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89437ecd-a9b6-4cdc-972c-3380593946f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.630679 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89437ecd-a9b6-4cdc-972c-3380593946f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:20.632365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89437ecd-a9b6-4cdc-972c-3380593946f4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:20.643112 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1357/2695] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:20.647406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87be4-f946-4961-9b4c-681393538b7b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:20.657386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb87be4-f946-4961-9b4c-681393538b7b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:20.661918 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75573a-d762-4e3e-a9b6-d7d02d599b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:20.662631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75573a-d762-4e3e-a9b6-d7d02d599b74 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.662860 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75573a-d762-4e3e-a9b6-d7d02d599b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.663140 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75573a-d762-4e3e-a9b6-d7d02d599b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.663885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75573a-d762-4e3e-a9b6-d7d02d599b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:20.665488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75573a-d762-4e3e-a9b6-d7d02d599b74 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:20.676509 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1339/2696] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:20.680259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89437ecd-a9b6-4cdc-972c-3380593946f4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:20.691447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89437ecd-a9b6-4cdc-972c-3380593946f4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:20.696143 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:20.697212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.697376 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.697648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] PATH_INFO: `/v3/projects/2407b8cdd7a14ce39f6eba4d39e5274f/users/d2c95dd407a54cf6b4f7934830643a10/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.713244 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2407b8cdd7a14ce39f6eba4d39e5274f, user_id=d2c95dd407a54cf6b4f7934830643a10, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:20.716289 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:20.879885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1358/2697] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:20 2022] PUT /identity/v3/projects/2407b8cdd7a14ce39f6eba4d39e5274f/users/d2c95dd407a54cf6b4f7934830643a10/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:04:20.890066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816648b0-c43e-4960-a986-80300f15891e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:20.890396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816648b0-c43e-4960-a986-80300f15891e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:20.890682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816648b0-c43e-4960-a986-80300f15891e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:20.917813 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-816648b0-c43e-4960-a986-80300f15891e None None] MFA Rules not processed for user `d2c95dd407a54cf6b4f7934830643a10`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:20.918971 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-816648b0-c43e-4960-a986-80300f15891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:20.984574 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1340/2698] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:20.996144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:20.997898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:21.012089 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:21.060682 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102b03-d482-4c5f-b271-c28a5520d4a0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:21.065781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df7299-3607-4fb4-9e80-913d45413760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:21.066707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7299-3607-4fb4-9e80-913d45413760 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:21.067124 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7299-3607-4fb4-9e80-913d45413760 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:21.067570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df7299-3607-4fb4-9e80-913d45413760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:21.080553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df7299-3607-4fb4-9e80-913d45413760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:21.083190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df7299-3607-4fb4-9e80-913d45413760 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:21.100508 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1359/2699] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:29.833639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816648b0-c43e-4960-a986-80300f15891e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:29.835406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-816648b0-c43e-4960-a986-80300f15891e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:29.842128 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:29.899315 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816648b0-c43e-4960-a986-80300f15891e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:29.908169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:29.909123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:29.909352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:29.909683 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] PATH_INFO: `/v3/users/2a8ce913e96840f79ec50b9077fddf56` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:29.910817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8ce913e96840f79ec50b9077fddf56)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:29.912912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:30.188885 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8ce913e96840f79ec50b9077fddf56'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:30.202556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8ce913e96840f79ec50b9077fddf56'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:30.203499 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8ce913e96840f79ec50b9077fddf56'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:30.204224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8ce913e96840f79ec50b9077fddf56'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:30.211120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8ce913e96840f79ec50b9077fddf56'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:30.310727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1341/2700] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:29 2022] DELETE /identity/v3/users/2a8ce913e96840f79ec50b9077fddf56 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:30.315155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeecd23-db3d-408d-afa3-2a2d537c77a0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:30.315680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeecd23-db3d-408d-afa3-2a2d537c77a0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:30.315785 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeecd23-db3d-408d-afa3-2a2d537c77a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:30.332940 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ceeecd23-db3d-408d-afa3-2a2d537c77a0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:30.334201 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ceeecd23-db3d-408d-afa3-2a2d537c77a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:30.395064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1360/2701] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:30.405276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:30.406751 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:30.475996 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b60a63-d2df-477e-b7d8-abe35ae850f6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:30.480575 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137389d8-f84c-480c-b8b6-03f4186e2146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:30.481253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137389d8-f84c-480c-b8b6-03f4186e2146 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:30.481524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137389d8-f84c-480c-b8b6-03f4186e2146 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:30.481817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137389d8-f84c-480c-b8b6-03f4186e2146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:30.492000 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137389d8-f84c-480c-b8b6-03f4186e2146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:30.493622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137389d8-f84c-480c-b8b6-03f4186e2146 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:30.503695 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1342/2702] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:30.941151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeecd23-db3d-408d-afa3-2a2d537c77a0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:30.942672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ceeecd23-db3d-408d-afa3-2a2d537c77a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:30.955912 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeecd23-db3d-408d-afa3-2a2d537c77a0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:30.960612 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:30.961486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:30.961771 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:30.962048 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] PATH_INFO: `/v3/projects/27c4c6ee062b4ad68484f090bfa8c70b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:30.963271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c4c6ee062b4ad68484f090bfa8c70b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:30.965494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:31.134106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c4c6ee062b4ad68484f090bfa8c70b', 'request_id': 'req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.139602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c4c6ee062b4ad68484f090bfa8c70b', 'request_id': 'req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.206991 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c4c6ee062b4ad68484f090bfa8c70b', 'request_id': 'req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.218766 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] The token cache is being invalidate because project 27c4c6ee062b4ad68484f090bfa8c70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:31.219062 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.219725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.221326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1361/2703] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:30 2022] DELETE /identity/v3/projects/27c4c6ee062b4ad68484f090bfa8c70b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:31.225326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137389d8-f84c-480c-b8b6-03f4186e2146 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:31.236154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137389d8-f84c-480c-b8b6-03f4186e2146 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:31.264862 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:31.297801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:31.298682 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:31.299154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:31.299788 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] PATH_INFO: `/v3/users/e4006c81462b4255871462c9d751bcc7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:31.300980 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4006c81462b4255871462c9d751bcc7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:31.302984 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:31.463501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4006c81462b4255871462c9d751bcc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.475713 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4006c81462b4255871462c9d751bcc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.476382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4006c81462b4255871462c9d751bcc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.476995 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4006c81462b4255871462c9d751bcc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.485233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a1491c6-55f7-4b09-9e6d-782e3ac8f559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4006c81462b4255871462c9d751bcc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:31.558498 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1343/2704] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:31 2022] DELETE /identity/v3/users/e4006c81462b4255871462c9d751bcc7 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:31.985620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:31.987095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:32.001974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b53249-e5ac-4e3d-93d0-6ca4ec8a2d99 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:32.045232 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:32.063673 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:32.064585 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.064865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.065141 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] PATH_INFO: `/v3/projects/62b5c0eb27dd471ebb3ec9e12fa563fe` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.066400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b5c0eb27dd471ebb3ec9e12fa563fe)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:32.068350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:32.220634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b5c0eb27dd471ebb3ec9e12fa563fe', 'request_id': 'req-13e00a05-5b94-41f2-adf6-1d93ac206a8a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:32.226040 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b5c0eb27dd471ebb3ec9e12fa563fe', 'request_id': 'req-13e00a05-5b94-41f2-adf6-1d93ac206a8a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:32.304237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b5c0eb27dd471ebb3ec9e12fa563fe', 'request_id': 'req-13e00a05-5b94-41f2-adf6-1d93ac206a8a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:32.316664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] The token cache is being invalidate because project 62b5c0eb27dd471ebb3ec9e12fa563fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:32.316955 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:32.317603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-13e00a05-5b94-41f2-adf6-1d93ac206a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:32.319159 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1362/2705] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:31 2022] DELETE /identity/v3/projects/62b5c0eb27dd471ebb3ec9e12fa563fe => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:32.324653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362898d-311f-480d-93e4-d9d12a8800f9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.325127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362898d-311f-480d-93e4-d9d12a8800f9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.325222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362898d-311f-480d-93e4-d9d12a8800f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.344661 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b362898d-311f-480d-93e4-d9d12a8800f9 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:32.345884 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b362898d-311f-480d-93e4-d9d12a8800f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:32.409453 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1344/2706] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:32.417968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af5412-039f-4d5a-8b50-03ff3257b774 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.418266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af5412-039f-4d5a-8b50-03ff3257b774 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.418551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af5412-039f-4d5a-8b50-03ff3257b774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.437430 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-56af5412-039f-4d5a-8b50-03ff3257b774 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:32.438677 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-56af5412-039f-4d5a-8b50-03ff3257b774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:32.446695 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1363/2707] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:32.451490 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362898d-311f-480d-93e4-d9d12a8800f9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:32.461113 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362898d-311f-480d-93e4-d9d12a8800f9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:32.465989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:32.466637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.466925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.467204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.467953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:04:32.468264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:32.469910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:32.481310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1345/2708] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:04:32.485409 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af5412-039f-4d5a-8b50-03ff3257b774 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:32.495203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af5412-039f-4d5a-8b50-03ff3257b774 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:32.500090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c22dbe-f9e0-46e7-be53-3c2aa9b6a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:32.500744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c22dbe-f9e0-46e7-be53-3c2aa9b6a03d admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.501034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c22dbe-f9e0-46e7-be53-3c2aa9b6a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.501316 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c22dbe-f9e0-46e7-be53-3c2aa9b6a03d admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.501996 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c22dbe-f9e0-46e7-be53-3c2aa9b6a03d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:32.503987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c22dbe-f9e0-46e7-be53-3c2aa9b6a03d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:32.591593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1364/2709] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:32 2022] POST /identity/v3/projects => generated 409 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:32.595529 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:32.606204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ba359-5972-44d5-b3f4-1857f13c1b6c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:32.635729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:32.661475 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a6014-f4ad-4ea6-b00c-e7f7dc293f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:32.662147 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a6014-f4ad-4ea6-b00c-e7f7dc293f44 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.662433 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a6014-f4ad-4ea6-b00c-e7f7dc293f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.662711 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a6014-f4ad-4ea6-b00c-e7f7dc293f44 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.663491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a6014-f4ad-4ea6-b00c-e7f7dc293f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:32.665416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a6014-f4ad-4ea6-b00c-e7f7dc293f44 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:32.784120 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1346/2710] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:32 2022] POST /identity/v3/users => generated 305 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:32.788277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c22dbe-f9e0-46e7-be53-3c2aa9b6a03d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:32.799034 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c22dbe-f9e0-46e7-be53-3c2aa9b6a03d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:32.803649 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfde1c-ae95-4eab-a024-d9ca422eeac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:32.804296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfde1c-ae95-4eab-a024-d9ca422eeac6 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.804610 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfde1c-ae95-4eab-a024-d9ca422eeac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.804889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfde1c-ae95-4eab-a024-d9ca422eeac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.805596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfde1c-ae95-4eab-a024-d9ca422eeac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:32.807858 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfde1c-ae95-4eab-a024-d9ca422eeac6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:32.818015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1365/2711] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:04:32.821934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a6014-f4ad-4ea6-b00c-e7f7dc293f44 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:32.831886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a6014-f4ad-4ea6-b00c-e7f7dc293f44 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:32.836465 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b54aa-baa9-4cdb-a774-99241cdb37cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:32.837124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b54aa-baa9-4cdb-a774-99241cdb37cd admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.837416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b54aa-baa9-4cdb-a774-99241cdb37cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.837690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b54aa-baa9-4cdb-a774-99241cdb37cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.838403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b54aa-baa9-4cdb-a774-99241cdb37cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:32.840300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b54aa-baa9-4cdb-a774-99241cdb37cd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:32.850161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1347/2712] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:32.854668 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfde1c-ae95-4eab-a024-d9ca422eeac6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:32.864678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfde1c-ae95-4eab-a024-d9ca422eeac6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:32.869888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c59b8-0c6d-4231-aaab-c9da23a75cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:32.870769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c59b8-0c6d-4231-aaab-c9da23a75cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:32.871060 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c59b8-0c6d-4231-aaab-c9da23a75cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:32.871375 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c59b8-0c6d-4231-aaab-c9da23a75cc3 admin admin] PATH_INFO: `/v3/projects/a116b5e831e14c7aaa1de868aed680e6/users/9b7756ddf57040d2aad436c2931cad2e/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:32.885890 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c59b8-0c6d-4231-aaab-c9da23a75cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a116b5e831e14c7aaa1de868aed680e6, user_id=9b7756ddf57040d2aad436c2931cad2e, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:32.888618 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c59b8-0c6d-4231-aaab-c9da23a75cc3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:33.003247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1366/2713] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:32 2022] PUT /identity/v3/projects/a116b5e831e14c7aaa1de868aed680e6/users/9b7756ddf57040d2aad436c2931cad2e/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:33.013890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf463-cf32-4d9f-816a-d5c86751e3b6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:33.014190 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf463-cf32-4d9f-816a-d5c86751e3b6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:33.014504 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf463-cf32-4d9f-816a-d5c86751e3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:33.043193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-46dcf463-cf32-4d9f-816a-d5c86751e3b6 None None] MFA Rules not processed for user `9b7756ddf57040d2aad436c2931cad2e`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:33.044487 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-46dcf463-cf32-4d9f-816a-d5c86751e3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:33.115934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1348/2714] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:33.120942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c59b8-0c6d-4231-aaab-c9da23a75cc3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:33.188172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c59b8-0c6d-4231-aaab-c9da23a75cc3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:33.196277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ccbb2-40db-43f0-baf5-aca2f899e3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:33.197438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ccbb2-40db-43f0-baf5-aca2f899e3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:33.197961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ccbb2-40db-43f0-baf5-aca2f899e3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:33.198476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ccbb2-40db-43f0-baf5-aca2f899e3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:33.199926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ccbb2-40db-43f0-baf5-aca2f899e3b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:33.203690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ccbb2-40db-43f0-baf5-aca2f899e3b0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:33.220891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:33.376795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1367/2715] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:04:33 2022] POST /identity/v3/projects => generated 407 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:33.379895 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf463-cf32-4d9f-816a-d5c86751e3b6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:33.393209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf463-cf32-4d9f-816a-d5c86751e3b6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:33.453569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c18c5-2247-4a93-9066-45c29d8e20eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:33.454283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c18c5-2247-4a93-9066-45c29d8e20eb admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:33.454530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c18c5-2247-4a93-9066-45c29d8e20eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:33.454806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c18c5-2247-4a93-9066-45c29d8e20eb admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:33.455512 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c18c5-2247-4a93-9066-45c29d8e20eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:33.457442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c18c5-2247-4a93-9066-45c29d8e20eb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:33.584028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1349/2716] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:04:33 2022] POST /identity/v3/users => generated 310 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:04:33.588740 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ccbb2-40db-43f0-baf5-aca2f899e3b0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:33.599438 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ccbb2-40db-43f0-baf5-aca2f899e3b0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:33.603976 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a18cc7-8d65-4a2b-881f-8a5e35240c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:33.604629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a18cc7-8d65-4a2b-881f-8a5e35240c7b admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:33.604919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a18cc7-8d65-4a2b-881f-8a5e35240c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:33.605206 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a18cc7-8d65-4a2b-881f-8a5e35240c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:33.605928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a18cc7-8d65-4a2b-881f-8a5e35240c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:33.607565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a18cc7-8d65-4a2b-881f-8a5e35240c7b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:33.618135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1368/2717] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:33.622196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c18c5-2247-4a93-9066-45c29d8e20eb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:33.632303 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c18c5-2247-4a93-9066-45c29d8e20eb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:33.637481 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a282a1c-0a2f-43e9-9794-87425baf2d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:33.638349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a282a1c-0a2f-43e9-9794-87425baf2d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:33.638636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a282a1c-0a2f-43e9-9794-87425baf2d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:33.638915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a282a1c-0a2f-43e9-9794-87425baf2d64 admin admin] PATH_INFO: `/v3/projects/163afd986b1d41bfb89accb639e4406d/users/6b19c5222fd54839a2a67d9e1f1cf9bf/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:33.654176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a282a1c-0a2f-43e9-9794-87425baf2d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163afd986b1d41bfb89accb639e4406d, user_id=6b19c5222fd54839a2a67d9e1f1cf9bf, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:33.657043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a282a1c-0a2f-43e9-9794-87425baf2d64 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:33.747375 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1350/2718] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:33 2022] PUT /identity/v3/projects/163afd986b1d41bfb89accb639e4406d/users/6b19c5222fd54839a2a67d9e1f1cf9bf/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:33.751195 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a18cc7-8d65-4a2b-881f-8a5e35240c7b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:33.761700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a18cc7-8d65-4a2b-881f-8a5e35240c7b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:33.816390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1788a5a-f56f-4ab0-80e9-3c7d1141d7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:33.817052 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788a5a-f56f-4ab0-80e9-3c7d1141d7eb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:33.817342 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788a5a-f56f-4ab0-80e9-3c7d1141d7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:33.817620 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788a5a-f56f-4ab0-80e9-3c7d1141d7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:33.818330 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1788a5a-f56f-4ab0-80e9-3c7d1141d7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:33.819962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1788a5a-f56f-4ab0-80e9-3c7d1141d7eb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:33.830187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1369/2719] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:04:33.834302 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a282a1c-0a2f-43e9-9794-87425baf2d64 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:33.844968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a282a1c-0a2f-43e9-9794-87425baf2d64 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:33.849690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:33.850591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:33.850907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:33.851188 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] PATH_INFO: `/v3/projects/163afd986b1d41bfb89accb639e4406d/users/6b19c5222fd54839a2a67d9e1f1cf9bf/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:33.853215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163afd986b1d41bfb89accb639e4406d, user_id=6b19c5222fd54839a2a67d9e1f1cf9bf, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:33.856003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:33.858218 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:33.866547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1351/2720] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:04:33 2022] PUT /identity/v3/projects/163afd986b1d41bfb89accb639e4406d/users/6b19c5222fd54839a2a67d9e1f1cf9bf/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:33.879472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598153-4454-4516-9d8a-f5c86f9956a4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:33.879764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598153-4454-4516-9d8a-f5c86f9956a4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:33.880053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a598153-4454-4516-9d8a-f5c86f9956a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:33.907090 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4a598153-4454-4516-9d8a-f5c86f9956a4 None None] MFA Rules not processed for user `6b19c5222fd54839a2a67d9e1f1cf9bf`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:33.908420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4a598153-4454-4516-9d8a-f5c86f9956a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:33.977268 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1370/2721] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:33.988253 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:33.989949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:34.046025 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40603bc3-2ab0-44cb-8603-be246f4a8fb8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:34.049750 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb76b7-489c-49b5-b1de-28349dd67135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:34.050385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb76b7-489c-49b5-b1de-28349dd67135 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:34.050672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb76b7-489c-49b5-b1de-28349dd67135 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:34.050950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb76b7-489c-49b5-b1de-28349dd67135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:34.060178 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb76b7-489c-49b5-b1de-28349dd67135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:34.061619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb76b7-489c-49b5-b1de-28349dd67135 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:34.070932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1352/2722] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:33 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:34.567722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598153-4454-4516-9d8a-f5c86f9956a4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:34.578274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a598153-4454-4516-9d8a-f5c86f9956a4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:34.581886 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:34.582532 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:34.582821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:34.583102 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:34.644775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:34.646230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:34.650902 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:34.655518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1371/2723] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:44.066798 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:50.067955 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bed1-1231-4c14-be39-d368907ffa1f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:50.068572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bed1-1231-4c14-be39-d368907ffa1f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:50.068572 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bed1-1231-4c14-be39-d368907ffa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:50.086296 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-14a1bed1-1231-4c14-be39-d368907ffa1f None None] MFA Rules not processed for user `cd49fd5c3cd5463b88efa5937d88cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:50.087551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-14a1bed1-1231-4c14-be39-d368907ffa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:50.146496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1353/2724] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:50.834058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:50.835624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:50.914365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303323a2-47ad-419a-b2c5-79ee24763bc9 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:50.919588 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397fd42-2b3c-4280-96ae-01d2f2eede2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:50.924450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397fd42-2b3c-4280-96ae-01d2f2eede2f service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:50.924450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397fd42-2b3c-4280-96ae-01d2f2eede2f service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:50.924450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397fd42-2b3c-4280-96ae-01d2f2eede2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:50.924450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b397fd42-2b3c-4280-96ae-01d2f2eede2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:50.934622 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397fd42-2b3c-4280-96ae-01d2f2eede2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:50.936225 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397fd42-2b3c-4280-96ae-01d2f2eede2f service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:50.946086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1372/2725] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:52.067710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1bed1-1231-4c14-be39-d368907ffa1f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:52.129558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1bed1-1231-4c14-be39-d368907ffa1f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:52.135022 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:52.135939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:52.136227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:52.136590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] PATH_INFO: `/v3/users/9b7756ddf57040d2aad436c2931cad2e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:52.137861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b7756ddf57040d2aad436c2931cad2e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:52.139846 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:52.377374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b7756ddf57040d2aad436c2931cad2e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:52.389952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b7756ddf57040d2aad436c2931cad2e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:52.390710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b7756ddf57040d2aad436c2931cad2e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:52.391409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b7756ddf57040d2aad436c2931cad2e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:52.398373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b7756ddf57040d2aad436c2931cad2e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:52.498546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1354/2726] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:52 2022] DELETE /identity/v3/users/9b7756ddf57040d2aad436c2931cad2e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:52.503118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e30d-b270-4447-8a02-e4d0b25e754c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:52.503638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e30d-b270-4447-8a02-e4d0b25e754c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:52.503742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652e30d-b270-4447-8a02-e4d0b25e754c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:52.521447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4652e30d-b270-4447-8a02-e4d0b25e754c None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:04:52.522652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4652e30d-b270-4447-8a02-e4d0b25e754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:52.580919 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1373/2727] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:04:52.591709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:52.593118 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:52.624245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:52.659901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a698fc3-8fa7-40cf-900d-3c9a169c5d90 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:52.663828 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434874d-993c-479b-b871-cff0ba5cf245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:52.664637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434874d-993c-479b-b871-cff0ba5cf245 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:52.664923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434874d-993c-479b-b871-cff0ba5cf245 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:52.665199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434874d-993c-479b-b871-cff0ba5cf245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:52.675452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434874d-993c-479b-b871-cff0ba5cf245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:52.677715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434874d-993c-479b-b871-cff0ba5cf245 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:52.686856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1355/2728] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:04:53.100415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652e30d-b270-4447-8a02-e4d0b25e754c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:53.102746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4652e30d-b270-4447-8a02-e4d0b25e754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:53.117827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652e30d-b270-4447-8a02-e4d0b25e754c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:53.124603 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:53.125348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:53.125624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:53.126139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] PATH_INFO: `/v3/projects/a116b5e831e14c7aaa1de868aed680e6` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:53.127253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a116b5e831e14c7aaa1de868aed680e6)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:53.129599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:53.259806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a116b5e831e14c7aaa1de868aed680e6', 'request_id': 'req-174b533f-590e-4ece-bcce-f68a1e60130e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.270064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a116b5e831e14c7aaa1de868aed680e6', 'request_id': 'req-174b533f-590e-4ece-bcce-f68a1e60130e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.360936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a116b5e831e14c7aaa1de868aed680e6', 'request_id': 'req-174b533f-590e-4ece-bcce-f68a1e60130e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.373167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] The token cache is being invalidate because project a116b5e831e14c7aaa1de868aed680e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:53.373457 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.374085 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.375684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1374/2729] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:53 2022] DELETE /identity/v3/projects/a116b5e831e14c7aaa1de868aed680e6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:53.379913 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434874d-993c-479b-b871-cff0ba5cf245 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:53.389857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434874d-993c-479b-b871-cff0ba5cf245 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:53.448221 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:53.449094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:53.449380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:53.461799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] PATH_INFO: `/v3/users/6b19c5222fd54839a2a67d9e1f1cf9bf` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:53.461799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b19c5222fd54839a2a67d9e1f1cf9bf)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:53.461799 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:53.672222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b19c5222fd54839a2a67d9e1f1cf9bf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.683950 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b19c5222fd54839a2a67d9e1f1cf9bf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.684636 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b19c5222fd54839a2a67d9e1f1cf9bf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.685229 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b19c5222fd54839a2a67d9e1f1cf9bf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.692371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b19c5222fd54839a2a67d9e1f1cf9bf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:53.773123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1356/2730] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:04:53 2022] DELETE /identity/v3/users/6b19c5222fd54839a2a67d9e1f1cf9bf => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:54.134806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:54.136936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:54.152556 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b533f-590e-4ece-bcce-f68a1e60130e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:54.163030 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:54.211095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:54.212215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:54.212729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:54.213021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] PATH_INFO: `/v3/projects/163afd986b1d41bfb89accb639e4406d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:54.215028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=163afd986b1d41bfb89accb639e4406d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:54.217070 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:54.361504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163afd986b1d41bfb89accb639e4406d', 'request_id': 'req-16227f48-30e6-4bd6-99e1-375a019d0f1f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:54.367051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163afd986b1d41bfb89accb639e4406d', 'request_id': 'req-16227f48-30e6-4bd6-99e1-375a019d0f1f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:54.461129 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163afd986b1d41bfb89accb639e4406d', 'request_id': 'req-16227f48-30e6-4bd6-99e1-375a019d0f1f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:54.473672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] The token cache is being invalidate because project 163afd986b1d41bfb89accb639e4406d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:04:54.473951 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:54.474594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:04:54.476259 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1375/2731] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:04:54 2022] DELETE /identity/v3/projects/163afd986b1d41bfb89accb639e4406d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:04:57.364942 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:04:57.366612 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:57.418251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:04:57.444064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa75712-2510-4272-8ed8-327187e9770e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:04:57.513555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf1f3d-0dab-4b76-a48f-fe5792067715 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:04:57.515029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf1f3d-0dab-4b76-a48f-fe5792067715 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:04:57.515527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf1f3d-0dab-4b76-a48f-fe5792067715 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:04:57.515913 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf1f3d-0dab-4b76-a48f-fe5792067715 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:04:57.518081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5acf1f3d-0dab-4b76-a48f-fe5792067715 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:04:57.531234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf1f3d-0dab-4b76-a48f-fe5792067715 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:04:57.533488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf1f3d-0dab-4b76-a48f-fe5792067715 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:04:57.544321 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1357/2732] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:04:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:05:00.752314 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:00.753778 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:00.818658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16227f48-30e6-4bd6-99e1-375a019d0f1f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:00.823122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:00.824022 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:00.824304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:00.824575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] PATH_INFO: `/v3/users/d2c95dd407a54cf6b4f7934830643a10` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:00.825748 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c95dd407a54cf6b4f7934830643a10)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:00.827699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:01.038856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:05:01.040566 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c95dd407a54cf6b4f7934830643a10'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:01.053348 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c95dd407a54cf6b4f7934830643a10'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:01.054003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c95dd407a54cf6b4f7934830643a10'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:01.054616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c95dd407a54cf6b4f7934830643a10'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:01.062055 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c95dd407a54cf6b4f7934830643a10'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:01.243950 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1376/2733] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:05:00 2022] DELETE /identity/v3/users/d2c95dd407a54cf6b4f7934830643a10 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:05:01.248466 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122036f0-2e86-42e3-87e4-60e364d8981d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:01.249002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122036f0-2e86-42e3-87e4-60e364d8981d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:01.249163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122036f0-2e86-42e3-87e4-60e364d8981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:01.265789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-122036f0-2e86-42e3-87e4-60e364d8981d None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:05:01.266997 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-122036f0-2e86-42e3-87e4-60e364d8981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:01.325863 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1358/2734] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:05:01.335593 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:01.337245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:01.399007 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d9411-5994-4d51-92c6-0224e3c3ace7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:01.403765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468412c-cc30-4da9-ad23-b9f97acde5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:01.404432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468412c-cc30-4da9-ad23-b9f97acde5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:01.404722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468412c-cc30-4da9-ad23-b9f97acde5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:01.405025 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468412c-cc30-4da9-ad23-b9f97acde5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:01.415004 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468412c-cc30-4da9-ad23-b9f97acde5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:01.416631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468412c-cc30-4da9-ad23-b9f97acde5b1 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:01.427400 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1377/2735] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:05:01.892255 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122036f0-2e86-42e3-87e4-60e364d8981d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:01.893940 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-122036f0-2e86-42e3-87e4-60e364d8981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:01.908327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122036f0-2e86-42e3-87e4-60e364d8981d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:01.914546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:01.915452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:01.915744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:01.916027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] PATH_INFO: `/v3/projects/2407b8cdd7a14ce39f6eba4d39e5274f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:01.917618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2407b8cdd7a14ce39f6eba4d39e5274f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:01.919601 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:02.036665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2407b8cdd7a14ce39f6eba4d39e5274f', 'request_id': 'req-3c217462-4aa0-4532-a65a-a88db99b5a4e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:02.042932 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2407b8cdd7a14ce39f6eba4d39e5274f', 'request_id': 'req-3c217462-4aa0-4532-a65a-a88db99b5a4e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:02.048638 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2407b8cdd7a14ce39f6eba4d39e5274f', 'request_id': 'req-3c217462-4aa0-4532-a65a-a88db99b5a4e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:02.051169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:05:02.139074 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] The token cache is being invalidate because project 2407b8cdd7a14ce39f6eba4d39e5274f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:05:02.139403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:02.140038 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3c217462-4aa0-4532-a65a-a88db99b5a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:05:02.141654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1359/2736] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:05:01 2022] DELETE /identity/v3/projects/2407b8cdd7a14ce39f6eba4d39e5274f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:05:02.146635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d8e2-ef3c-45f9-8195-1257fec19c59 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.147117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d8e2-ef3c-45f9-8195-1257fec19c59 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.147229 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628d8e2-ef3c-45f9-8195-1257fec19c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.166559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1628d8e2-ef3c-45f9-8195-1257fec19c59 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:05:02.167993 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1628d8e2-ef3c-45f9-8195-1257fec19c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:02.231239 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1378/2737] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:05:02.239793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d304de-ff71-478d-ae25-6467cf3e1d48 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.240100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d304de-ff71-478d-ae25-6467cf3e1d48 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.240373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d304de-ff71-478d-ae25-6467cf3e1d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.259410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c9d304de-ff71-478d-ae25-6467cf3e1d48 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:05:02.261486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c9d304de-ff71-478d-ae25-6467cf3e1d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:02.273267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1360/2738] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:05:02.276985 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628d8e2-ef3c-45f9-8195-1257fec19c59 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:02.286781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628d8e2-ef3c-45f9-8195-1257fec19c59 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:02.291536 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:02.292442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.292757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.293042 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.293765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:05:02.294083 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:02.295728 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:02.308547 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1379/2739] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:05:02.313280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d304de-ff71-478d-ae25-6467cf3e1d48 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:02.329023 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d304de-ff71-478d-ae25-6467cf3e1d48 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:02.333994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10dafc-00ce-4c54-b231-a77380b241d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:02.334665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10dafc-00ce-4c54-b231-a77380b241d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.334957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10dafc-00ce-4c54-b231-a77380b241d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.335240 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10dafc-00ce-4c54-b231-a77380b241d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.336626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10dafc-00ce-4c54-b231-a77380b241d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:02.338635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10dafc-00ce-4c54-b231-a77380b241d6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:02.424654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1361/2740] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:05:02 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:05:02.425808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:02.436067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3ba13-4470-46b5-a3b8-5ee0af419d90 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:02.484162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:05:02.492818 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0691b-ffba-4ef4-84ae-1c706f7855ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:02.493487 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0691b-ffba-4ef4-84ae-1c706f7855ec admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.493773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0691b-ffba-4ef4-84ae-1c706f7855ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.494053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0691b-ffba-4ef4-84ae-1c706f7855ec admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.494734 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0691b-ffba-4ef4-84ae-1c706f7855ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:02.496715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0691b-ffba-4ef4-84ae-1c706f7855ec admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:02.610199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1380/2741] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:05:02 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:05:02.614211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10dafc-00ce-4c54-b231-a77380b241d6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:02.625313 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10dafc-00ce-4c54-b231-a77380b241d6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:02.630009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f0d2e-9253-4d74-97d4-786f77adf82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:02.630768 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f0d2e-9253-4d74-97d4-786f77adf82d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.631065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f0d2e-9253-4d74-97d4-786f77adf82d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.631380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f0d2e-9253-4d74-97d4-786f77adf82d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.632108 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f0d2e-9253-4d74-97d4-786f77adf82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:02.633732 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f0d2e-9253-4d74-97d4-786f77adf82d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:02.645334 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1362/2742] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:05:02.649290 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0691b-ffba-4ef4-84ae-1c706f7855ec admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:02.659171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0691b-ffba-4ef4-84ae-1c706f7855ec admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:02.663815 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff67d38-6daa-4f2c-a173-882235435182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:02.664469 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67d38-6daa-4f2c-a173-882235435182 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.664755 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67d38-6daa-4f2c-a173-882235435182 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.665037 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67d38-6daa-4f2c-a173-882235435182 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.665751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff67d38-6daa-4f2c-a173-882235435182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:02.667369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff67d38-6daa-4f2c-a173-882235435182 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:02.677926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1381/2743] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:05:02.681956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f0d2e-9253-4d74-97d4-786f77adf82d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:02.692524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f0d2e-9253-4d74-97d4-786f77adf82d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:02.697124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:02.698053 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.698847 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.699133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] PATH_INFO: `/v3/projects/1678935d4a174049845ec25084e49f66/users/a797610e65d74f469b1f0c4f984a4463/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.713946 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1678935d4a174049845ec25084e49f66, user_id=a797610e65d74f469b1f0c4f984a4463, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:02.717007 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:02.871228 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1363/2744] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:05:02 2022] PUT /identity/v3/projects/1678935d4a174049845ec25084e49f66/users/a797610e65d74f469b1f0c4f984a4463/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:05:02.881777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd559-dbf7-4ae7-9179-7646fe15f6b4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:02.882075 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd559-dbf7-4ae7-9179-7646fe15f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:02.882358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd559-dbf7-4ae7-9179-7646fe15f6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:02.908880 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c40dd559-dbf7-4ae7-9179-7646fe15f6b4 None None] MFA Rules not processed for user `a797610e65d74f469b1f0c4f984a4463`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:05:02.910105 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c40dd559-dbf7-4ae7-9179-7646fe15f6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:02.973651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1382/2745] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:05:02.988139 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:02.989868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:03.046689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b5a45-02cb-45b4-9cc3-1fafb0aee3f1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:03.050307 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfba69-f81a-4e2a-9ad0-5f2cc2bcfdda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:03.050952 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfba69-f81a-4e2a-9ad0-5f2cc2bcfdda service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:03.051244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfba69-f81a-4e2a-9ad0-5f2cc2bcfdda service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:03.051562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfba69-f81a-4e2a-9ad0-5f2cc2bcfdda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:03.060814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfba69-f81a-4e2a-9ad0-5f2cc2bcfdda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:03.062261 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfba69-f81a-4e2a-9ad0-5f2cc2bcfdda service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:03.071072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1364/2746] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:05:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:05:03.887377 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd559-dbf7-4ae7-9179-7646fe15f6b4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:05:03.888936 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c40dd559-dbf7-4ae7-9179-7646fe15f6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:03.960249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd559-dbf7-4ae7-9179-7646fe15f6b4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:05:03.966598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2147ec-d462-4e89-81fd-febb0201fb38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:05:03.967241 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2147ec-d462-4e89-81fd-febb0201fb38 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:05:03.967572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2147ec-d462-4e89-81fd-febb0201fb38 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:05:03.967850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2147ec-d462-4e89-81fd-febb0201fb38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:05:03.969606 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8d2147ec-d462-4e89-81fd-febb0201fb38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:05:03.982047 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2147ec-d462-4e89-81fd-febb0201fb38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:05:03.983715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2147ec-d462-4e89-81fd-febb0201fb38 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:05:03.994630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1383/2747] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:05:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:05:13.066994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:05:13.989958 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:11.972415 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfba69-f81a-4e2a-9ad0-5f2cc2bcfdda service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:12.033337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfba69-f81a-4e2a-9ad0-5f2cc2bcfdda service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:12.037876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:12.038767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:12.039058 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:12.039374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] PATH_INFO: `/v3/users/a797610e65d74f469b1f0c4f984a4463` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:12.040570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a797610e65d74f469b1f0c4f984a4463)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:12.042523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:12.303919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a797610e65d74f469b1f0c4f984a4463'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:12.315929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a797610e65d74f469b1f0c4f984a4463'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:12.316608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a797610e65d74f469b1f0c4f984a4463'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:12.317217 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a797610e65d74f469b1f0c4f984a4463'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:12.324081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a797610e65d74f469b1f0c4f984a4463'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:12.443684 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1365/2748] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:06:11 2022] DELETE /identity/v3/users/a797610e65d74f469b1f0c4f984a4463 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:12.448389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8feeeb-ec06-4c2f-87ec-950db8014875 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:12.448891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8feeeb-ec06-4c2f-87ec-950db8014875 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:12.448995 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8feeeb-ec06-4c2f-87ec-950db8014875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:12.467988 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cd8feeeb-ec06-4c2f-87ec-950db8014875 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:12.469213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd8feeeb-ec06-4c2f-87ec-950db8014875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:12.544634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1384/2749] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:06:12.554693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:12.556102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:12.612687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760af2b0-e228-4844-845c-d74890a4f417 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:12.616336 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf04e30-a083-4048-a7dc-209ff3943771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:12.616982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04e30-a083-4048-a7dc-209ff3943771 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:12.617271 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04e30-a083-4048-a7dc-209ff3943771 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:12.617553 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04e30-a083-4048-a7dc-209ff3943771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:12.626645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf04e30-a083-4048-a7dc-209ff3943771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:12.628159 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf04e30-a083-4048-a7dc-209ff3943771 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:12.637831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1366/2750] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:13.056402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8feeeb-ec06-4c2f-87ec-950db8014875 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:13.057814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd8feeeb-ec06-4c2f-87ec-950db8014875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:13.070583 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8feeeb-ec06-4c2f-87ec-950db8014875 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:13.075264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:13.076154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.076443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.076736 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] PATH_INFO: `/v3/projects/1678935d4a174049845ec25084e49f66` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.080494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1678935d4a174049845ec25084e49f66)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:13.080494 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:13.208183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1678935d4a174049845ec25084e49f66', 'request_id': 'req-0f4da82f-ae5a-4744-bfed-227b9aef4665'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:13.213693 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1678935d4a174049845ec25084e49f66', 'request_id': 'req-0f4da82f-ae5a-4744-bfed-227b9aef4665'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:13.287386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1678935d4a174049845ec25084e49f66', 'request_id': 'req-0f4da82f-ae5a-4744-bfed-227b9aef4665'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:13.298813 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] The token cache is being invalidate because project 1678935d4a174049845ec25084e49f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:06:13.299099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:13.299768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f4da82f-ae5a-4744-bfed-227b9aef4665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:13.301305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1385/2751] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:06:13 2022] DELETE /identity/v3/projects/1678935d4a174049845ec25084e49f66 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:13.306669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d577210-7fb3-4b9c-b49f-6bc77288a668 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.307164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d577210-7fb3-4b9c-b49f-6bc77288a668 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.307266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d577210-7fb3-4b9c-b49f-6bc77288a668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.324514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4d577210-7fb3-4b9c-b49f-6bc77288a668 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:13.326232 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4d577210-7fb3-4b9c-b49f-6bc77288a668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:13.343483 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:13.385901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1367/2752] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:13.448846 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebb862-b679-4130-8d87-cf0f1cc6b284 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.449398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebb862-b679-4130-8d87-cf0f1cc6b284 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.449398 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebb862-b679-4130-8d87-cf0f1cc6b284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.471235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-26ebb862-b679-4130-8d87-cf0f1cc6b284 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:13.472486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-26ebb862-b679-4130-8d87-cf0f1cc6b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:13.480534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1386/2753] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:13.485065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d577210-7fb3-4b9c-b49f-6bc77288a668 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:13.494758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d577210-7fb3-4b9c-b49f-6bc77288a668 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:13.499330 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:13.500027 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.500312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.500587 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.501301 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:06:13.501626 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:13.503207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:13.515444 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1368/2754] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:06:13.519821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebb862-b679-4130-8d87-cf0f1cc6b284 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:13.529812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebb862-b679-4130-8d87-cf0f1cc6b284 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:13.534517 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf7f3d-98f8-4e80-96f4-b1d59e0343bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:13.535170 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf7f3d-98f8-4e80-96f4-b1d59e0343bb admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.535489 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf7f3d-98f8-4e80-96f4-b1d59e0343bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.535774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf7f3d-98f8-4e80-96f4-b1d59e0343bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.536463 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf7f3d-98f8-4e80-96f4-b1d59e0343bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:13.538466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf7f3d-98f8-4e80-96f4-b1d59e0343bb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:13.632452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1387/2755] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:06:13 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:13.636474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:13.647276 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec83fb-55eb-4b7a-aa09-d00be6ae795f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:13.703907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a04eb-4176-46f5-a0c4-cff30e2e11c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:13.704578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a04eb-4176-46f5-a0c4-cff30e2e11c7 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.704868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a04eb-4176-46f5-a0c4-cff30e2e11c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.705151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a04eb-4176-46f5-a0c4-cff30e2e11c7 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.705851 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a04eb-4176-46f5-a0c4-cff30e2e11c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:13.707829 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a04eb-4176-46f5-a0c4-cff30e2e11c7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:13.818947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1369/2756] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:06:13 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:13.823205 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf7f3d-98f8-4e80-96f4-b1d59e0343bb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:13.834104 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf7f3d-98f8-4e80-96f4-b1d59e0343bb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:13.838760 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddef3de-37a6-4e8a-ba5b-6a5f310c8816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:13.840364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddef3de-37a6-4e8a-ba5b-6a5f310c8816 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.840364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddef3de-37a6-4e8a-ba5b-6a5f310c8816 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.840364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddef3de-37a6-4e8a-ba5b-6a5f310c8816 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.840831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddef3de-37a6-4e8a-ba5b-6a5f310c8816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:13.842684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddef3de-37a6-4e8a-ba5b-6a5f310c8816 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:13.853497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1388/2757] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:13.857639 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a04eb-4176-46f5-a0c4-cff30e2e11c7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:13.867508 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a04eb-4176-46f5-a0c4-cff30e2e11c7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:13.872104 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e511-b7a4-4f0a-b1e7-8de3ae6ff5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:13.872757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e511-b7a4-4f0a-b1e7-8de3ae6ff5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.873043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e511-b7a4-4f0a-b1e7-8de3ae6ff5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.873929 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e511-b7a4-4f0a-b1e7-8de3ae6ff5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.874781 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e511-b7a4-4f0a-b1e7-8de3ae6ff5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:13.877227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e511-b7a4-4f0a-b1e7-8de3ae6ff5f3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:13.893043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1370/2758] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:13.897202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddef3de-37a6-4e8a-ba5b-6a5f310c8816 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:13.907255 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddef3de-37a6-4e8a-ba5b-6a5f310c8816 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:13.912038 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:13.912937 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:13.913223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:13.913504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] PATH_INFO: `/v3/projects/801e210cf8c64e7184b178fbde47860c/users/14fecc08cd364a48a8640d522577aa39/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:13.929505 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801e210cf8c64e7184b178fbde47860c, user_id=14fecc08cd364a48a8640d522577aa39, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:13.932236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:14.002020 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1389/2759] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:13 2022] PUT /identity/v3/projects/801e210cf8c64e7184b178fbde47860c/users/14fecc08cd364a48a8640d522577aa39/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:14.007181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709e86-72e0-4ed9-b921-bf043d156e6e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:14.007578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709e86-72e0-4ed9-b921-bf043d156e6e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:14.010154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709e86-72e0-4ed9-b921-bf043d156e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:14.026279 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-28709e86-72e0-4ed9-b921-bf043d156e6e None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:14.027519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-28709e86-72e0-4ed9-b921-bf043d156e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:14.084730 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1371/2760] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:14.095058 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:14.096781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:14.118390 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:14.152436 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca7499-7c57-4c14-9eac-e21a812a0e29 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:14.156900 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:14.161134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:14.161134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:14.161134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:14.168162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:14.169599 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:14.179408 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1390/2761] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:16.522718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28709e86-72e0-4ed9-b921-bf043d156e6e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:16.524280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-28709e86-72e0-4ed9-b921-bf043d156e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:16.567957 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:16.639107 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28709e86-72e0-4ed9-b921-bf043d156e6e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:16.645622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:16.649156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:16.649156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:16.649156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:16.658174 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:16.750449 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:16.752156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:16.772385 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1372/2762] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:19.679031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:19.680651 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:19.766684 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f9323-e711-4597-80a4-4a222a357d60 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:19.772745 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8009f-fd3c-4719-8335-a1aa80cbcb98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:19.773565 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8009f-fd3c-4719-8335-a1aa80cbcb98 service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:19.773852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8009f-fd3c-4719-8335-a1aa80cbcb98 service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:19.774188 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8009f-fd3c-4719-8335-a1aa80cbcb98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:19.776023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2ae8009f-fd3c-4719-8335-a1aa80cbcb98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:19.788344 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8009f-fd3c-4719-8335-a1aa80cbcb98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:19.789982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8009f-fd3c-4719-8335-a1aa80cbcb98 service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:19.802267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1391/2763] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:19.922086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:19.924340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:19.955054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49264be-b47e-4d33-b8dd-0c161e195095 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:19.963619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:19.963619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:19.963619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:19.963619 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] PATH_INFO: `/v3/users/cd49fd5c3cd5463b88efa5937d88cea0` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:19.965610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd49fd5c3cd5463b88efa5937d88cea0)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:19.968445 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:20.238246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd49fd5c3cd5463b88efa5937d88cea0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.267337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd49fd5c3cd5463b88efa5937d88cea0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.267337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd49fd5c3cd5463b88efa5937d88cea0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.267337 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd49fd5c3cd5463b88efa5937d88cea0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.275089 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd49fd5c3cd5463b88efa5937d88cea0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.358410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dcb42-dbc7-4f58-8128-ab45cb9cd61a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:20.358710 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dcb42-dbc7-4f58-8128-ab45cb9cd61a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:20.359000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dcb42-dbc7-4f58-8128-ab45cb9cd61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:20.395798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-794dcb42-dbc7-4f58-8128-ab45cb9cd61a None None] MFA Rules not processed for user `14fecc08cd364a48a8640d522577aa39`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:20.397029 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-794dcb42-dbc7-4f58-8128-ab45cb9cd61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:20.414637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1373/2764] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:06:19 2022] DELETE /identity/v3/users/cd49fd5c3cd5463b88efa5937d88cea0 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:20.494162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1392/2765] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:06:20.501145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:20.503146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:20.515170 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:20.580216 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1155-4dcb-4f89-84ad-b9d39261413e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:20.585388 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02418a5-b41e-4de2-908d-872857d8e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:20.586064 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02418a5-b41e-4de2-908d-872857d8e898 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:20.586349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02418a5-b41e-4de2-908d-872857d8e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:20.586658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02418a5-b41e-4de2-908d-872857d8e898 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:20.587394 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02418a5-b41e-4de2-908d-872857d8e898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:20.589327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02418a5-b41e-4de2-908d-872857d8e898 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:20.705550 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dcb42-dbc7-4f58-8128-ab45cb9cd61a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:20.707266 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-794dcb42-dbc7-4f58-8128-ab45cb9cd61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:20.710014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1374/2766] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:06:20 2022] POST /identity/v3/projects => generated 377 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:20.715285 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02418a5-b41e-4de2-908d-872857d8e898 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:20.729350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dcb42-dbc7-4f58-8128-ab45cb9cd61a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:20.781932 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:20.800885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:20.801798 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:20.802086 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:20.802374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] PATH_INFO: `/v3/projects/836bc4ea37f841cf9704999d88a627c4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:20.803669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] RBAC: Authorizing `identity:delete_project(project_id=836bc4ea37f841cf9704999d88a627c4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:20.805597 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:20.812122 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02418a5-b41e-4de2-908d-872857d8e898 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:20.816888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8e068-9cc4-4cb6-85f9-cb7dd5f6b99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:20.819365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8e068-9cc4-4cb6-85f9-cb7dd5f6b99d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:20.819365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8e068-9cc4-4cb6-85f9-cb7dd5f6b99d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:20.819365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8e068-9cc4-4cb6-85f9-cb7dd5f6b99d admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:20.819365 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8e068-9cc4-4cb6-85f9-cb7dd5f6b99d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:20.820759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8e068-9cc4-4cb6-85f9-cb7dd5f6b99d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:20.891919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1375/2767] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:06:20 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:20.895812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8e068-9cc4-4cb6-85f9-cb7dd5f6b99d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:20.905356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836bc4ea37f841cf9704999d88a627c4', 'request_id': 'req-726e1c8b-de75-41fa-ad4a-fb60b20c205e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.907091 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8e068-9cc4-4cb6-85f9-cb7dd5f6b99d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:20.911318 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836bc4ea37f841cf9704999d88a627c4', 'request_id': 'req-726e1c8b-de75-41fa-ad4a-fb60b20c205e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.964963 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c64e3-8667-479b-931b-f90e408bc519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:20.965658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c64e3-8667-479b-931b-f90e408bc519 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:20.965962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c64e3-8667-479b-931b-f90e408bc519 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:20.966241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c64e3-8667-479b-931b-f90e408bc519 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:20.967026 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c64e3-8667-479b-931b-f90e408bc519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:20.969431 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c64e3-8667-479b-931b-f90e408bc519 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:20.969638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836bc4ea37f841cf9704999d88a627c4', 'request_id': 'req-726e1c8b-de75-41fa-ad4a-fb60b20c205e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.980175 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1376/2768] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:06:20.984086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c64e3-8667-479b-931b-f90e408bc519 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:20.984952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] The token cache is being invalidate because project 836bc4ea37f841cf9704999d88a627c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:06:20.985249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.985903 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-726e1c8b-de75-41fa-ad4a-fb60b20c205e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:20.987486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1393/2769] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:06:20 2022] DELETE /identity/v3/projects/836bc4ea37f841cf9704999d88a627c4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:20.992320 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c8d8d-e17c-4c74-8c74-7227218ef488 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:20.992613 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c8d8d-e17c-4c74-8c74-7227218ef488 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:20.992898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c8d8d-e17c-4c74-8c74-7227218ef488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:20.993800 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c64e3-8667-479b-931b-f90e408bc519 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:20.998473 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:20.999371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:20.999669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:20.999949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] PATH_INFO: `/v3/projects/1ae5a68107b04f41ba4bc3284d7ba322/users/b1f79c58c23e48ad89a8d05564fea50b/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.011347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-289c8d8d-e17c-4c74-8c74-7227218ef488 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:21.012917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-289c8d8d-e17c-4c74-8c74-7227218ef488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:21.035472 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1394/2770] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:06:21.041769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929904d-169e-4c66-9faa-0286d31765ac None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.042194 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929904d-169e-4c66-9faa-0286d31765ac None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.042404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929904d-169e-4c66-9faa-0286d31765ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.048765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae5a68107b04f41ba4bc3284d7ba322, user_id=b1f79c58c23e48ad89a8d05564fea50b, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.052754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.063223 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a929904d-169e-4c66-9faa-0286d31765ac None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:21.065123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a929904d-169e-4c66-9faa-0286d31765ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:21.073084 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1395/2771] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:06:21.077688 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929904d-169e-4c66-9faa-0286d31765ac None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:21.094539 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929904d-169e-4c66-9faa-0286d31765ac None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:21.099389 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:21.100053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.100341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.100627 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.101346 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:06:21.101723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.103388 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.115518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1396/2772] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 16:06:21.126106 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:21.136018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ab2f1-f7c8-441f-bb38-3adb6dca9930 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:21.141866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8256-4ad8-49bd-83cb-7ea77636c0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:21.142514 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8256-4ad8-49bd-83cb-7ea77636c0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.142802 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8256-4ad8-49bd-83cb-7ea77636c0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.143081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a8256-4ad8-49bd-83cb-7ea77636c0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.143814 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a8256-4ad8-49bd-83cb-7ea77636c0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.145716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a8256-4ad8-49bd-83cb-7ea77636c0e1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.257123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1377/2773] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:20 2022] PUT /identity/v3/projects/1ae5a68107b04f41ba4bc3284d7ba322/users/b1f79c58c23e48ad89a8d05564fea50b/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:06:21.271916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:21.271916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:21.311956 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1397/2774] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:06:21 2022] POST /identity/v3/projects => generated 399 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:06:21.318128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8256-4ad8-49bd-83cb-7ea77636c0e1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:21.347496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6faf04-da1d-43a1-95de-fdf25b0883cc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:21.402246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a8256-4ad8-49bd-83cb-7ea77636c0e1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:21.407904 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224e58-44e0-4d89-8392-31934b0ae27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:21.408808 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224e58-44e0-4d89-8392-31934b0ae27a admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.409317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224e58-44e0-4d89-8392-31934b0ae27a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.409962 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224e58-44e0-4d89-8392-31934b0ae27a admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.410821 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f224e58-44e0-4d89-8392-31934b0ae27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.412968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f224e58-44e0-4d89-8392-31934b0ae27a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.417326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0131a1-a508-4265-a2db-66133db567f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:21.418125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0131a1-a508-4265-a2db-66133db567f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.418478 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0131a1-a508-4265-a2db-66133db567f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.418826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0131a1-a508-4265-a2db-66133db567f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.419658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0131a1-a508-4265-a2db-66133db567f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.421875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0131a1-a508-4265-a2db-66133db567f5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.432374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1378/2775] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:06:21.436718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0131a1-a508-4265-a2db-66133db567f5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:21.447003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0131a1-a508-4265-a2db-66133db567f5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:21.452084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089fdb0-a67e-4e40-b81e-2f1967e54668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:21.452969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089fdb0-a67e-4e40-b81e-2f1967e54668 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.453317 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089fdb0-a67e-4e40-b81e-2f1967e54668 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.453595 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089fdb0-a67e-4e40-b81e-2f1967e54668 admin admin] PATH_INFO: `/v3/projects/1ae5a68107b04f41ba4bc3284d7ba322/users/b1f79c58c23e48ad89a8d05564fea50b/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.455679 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089fdb0-a67e-4e40-b81e-2f1967e54668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae5a68107b04f41ba4bc3284d7ba322, user_id=b1f79c58c23e48ad89a8d05564fea50b, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.458416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089fdb0-a67e-4e40-b81e-2f1967e54668 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.469264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1379/2776] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:21 2022] PUT /identity/v3/projects/1ae5a68107b04f41ba4bc3284d7ba322/users/b1f79c58c23e48ad89a8d05564fea50b/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:21.558665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1398/2777] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:06:21 2022] POST /identity/v3/users => generated 300 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:21.563251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089fdb0-a67e-4e40-b81e-2f1967e54668 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:21.625816 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089fdb0-a67e-4e40-b81e-2f1967e54668 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:21.633616 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669290f1-5807-4e6f-98a7-08b94456f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:21.634858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669290f1-5807-4e6f-98a7-08b94456f528 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.635160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669290f1-5807-4e6f-98a7-08b94456f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.635933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669290f1-5807-4e6f-98a7-08b94456f528 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.636417 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669290f1-5807-4e6f-98a7-08b94456f528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.638069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669290f1-5807-4e6f-98a7-08b94456f528 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.655329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1380/2778] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:21.659716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224e58-44e0-4d89-8392-31934b0ae27a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:21.671537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224e58-44e0-4d89-8392-31934b0ae27a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:21.676721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7c94c-fcea-4067-9395-893f29f9dfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:21.677450 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7c94c-fcea-4067-9395-893f29f9dfcb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.677856 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7c94c-fcea-4067-9395-893f29f9dfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.678063 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7c94c-fcea-4067-9395-893f29f9dfcb admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.678959 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7c94c-fcea-4067-9395-893f29f9dfcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.680699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7c94c-fcea-4067-9395-893f29f9dfcb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.697122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1399/2779] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:06:21.698628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669290f1-5807-4e6f-98a7-08b94456f528 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:21.709373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669290f1-5807-4e6f-98a7-08b94456f528 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:21.714374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:21.715316 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.715613 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.715890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] PATH_INFO: `/v3/projects/10f9c137005b4d6ea065c9f8ba27c198/users/c269917436694c68a614ecfc38b2df32/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.730945 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f9c137005b4d6ea065c9f8ba27c198, user_id=c269917436694c68a614ecfc38b2df32, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:21.734209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:21.752227 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:21.828617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1381/2780] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:21 2022] PUT /identity/v3/projects/10f9c137005b4d6ea065c9f8ba27c198/users/c269917436694c68a614ecfc38b2df32/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:21.839646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fada-2d2e-45bb-98d7-aac6ddfe021e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:21.840113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fada-2d2e-45bb-98d7-aac6ddfe021e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:21.840310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fada-2d2e-45bb-98d7-aac6ddfe021e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:21.847575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:22.064665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5755fada-2d2e-45bb-98d7-aac6ddfe021e None None] MFA Rules not processed for user `c269917436694c68a614ecfc38b2df32`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:22.066131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5755fada-2d2e-45bb-98d7-aac6ddfe021e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:22.160743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1400/2781] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:22.172181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:22.173969 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:22.232770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:22.282809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80368a-3e60-452b-afa4-e13cca185226 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:22.291208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:22.291208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:22.291208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:22.291208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:22.299017 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:22.301106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:22.325650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1382/2782] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:27.476171 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc2095-a624-47ec-98f1-ddf12c57ff74 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:27.476672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc2095-a624-47ec-98f1-ddf12c57ff74 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:27.476672 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc2095-a624-47ec-98f1-ddf12c57ff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:27.504131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-78bc2095-a624-47ec-98f1-ddf12c57ff74 None None] MFA Rules not processed for user `b1f79c58c23e48ad89a8d05564fea50b`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:27.505387 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-78bc2095-a624-47ec-98f1-ddf12c57ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:27.579520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1401/2783] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:27.596844 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:27.598384 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:27.677241 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48e0dc-a936-440f-b7e2-9ab437450360 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:27.683121 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94aeb1-9309-498f-96cd-49400e48323c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:27.683819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94aeb1-9309-498f-96cd-49400e48323c service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:27.684112 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94aeb1-9309-498f-96cd-49400e48323c service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:27.684396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94aeb1-9309-498f-96cd-49400e48323c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:27.686273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7b94aeb1-9309-498f-96cd-49400e48323c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:27.748207 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94aeb1-9309-498f-96cd-49400e48323c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:27.749953 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94aeb1-9309-498f-96cd-49400e48323c service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:27.760411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1383/2784] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:32.915652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc2095-a624-47ec-98f1-ddf12c57ff74 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:32.976680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc2095-a624-47ec-98f1-ddf12c57ff74 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:32.981140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:32.982003 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:32.982284 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:32.982586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] PATH_INFO: `/v3/users/c269917436694c68a614ecfc38b2df32` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:32.983981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c269917436694c68a614ecfc38b2df32)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:32.985885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:33.007419 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:33.216260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c269917436694c68a614ecfc38b2df32'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:33.228504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c269917436694c68a614ecfc38b2df32'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:33.229190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c269917436694c68a614ecfc38b2df32'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:33.229812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c269917436694c68a614ecfc38b2df32'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:33.236889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c269917436694c68a614ecfc38b2df32'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:33.371262 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1402/2785] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:06:32 2022] DELETE /identity/v3/users/c269917436694c68a614ecfc38b2df32 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:33.376931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85873-b9a3-46e1-a163-bd7796738cf9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:33.377411 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85873-b9a3-46e1-a163-bd7796738cf9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:33.377519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c85873-b9a3-46e1-a163-bd7796738cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:33.395661 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-12c85873-b9a3-46e1-a163-bd7796738cf9 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:33.399093 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-12c85873-b9a3-46e1-a163-bd7796738cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:33.435012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:33.461839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1384/2786] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:33.471901 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:33.473249 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:33.533193 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d60e5e-e665-4ff8-ad88-9005bb3b80eb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:33.536787 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511fa4e-b4a2-4635-9fd3-7760731c5534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:33.537427 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fa4e-b4a2-4635-9fd3-7760731c5534 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:33.537712 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fa4e-b4a2-4635-9fd3-7760731c5534 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:33.537994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511fa4e-b4a2-4635-9fd3-7760731c5534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:33.547643 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511fa4e-b4a2-4635-9fd3-7760731c5534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:33.549077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511fa4e-b4a2-4635-9fd3-7760731c5534 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:33.558475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1403/2787] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:06:34.045764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c85873-b9a3-46e1-a163-bd7796738cf9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:34.047447 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-12c85873-b9a3-46e1-a163-bd7796738cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:34.062259 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c85873-b9a3-46e1-a163-bd7796738cf9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:34.068264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:34.069205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.069710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.069814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] PATH_INFO: `/v3/projects/10f9c137005b4d6ea065c9f8ba27c198` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.071042 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f9c137005b4d6ea065c9f8ba27c198)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:34.073195 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:34.201739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f9c137005b4d6ea065c9f8ba27c198', 'request_id': 'req-515d4160-0926-42ac-bde1-50205de54a50'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:34.201739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f9c137005b4d6ea065c9f8ba27c198', 'request_id': 'req-515d4160-0926-42ac-bde1-50205de54a50'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:34.204146 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f9c137005b4d6ea065c9f8ba27c198', 'request_id': 'req-515d4160-0926-42ac-bde1-50205de54a50'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:34.340167 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] The token cache is being invalidate because project 10f9c137005b4d6ea065c9f8ba27c198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:06:34.340392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:34.340974 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-515d4160-0926-42ac-bde1-50205de54a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:34.342523 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1385/2788] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:06:34 2022] DELETE /identity/v3/projects/10f9c137005b4d6ea065c9f8ba27c198 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:34.347757 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c205b-af9b-4efd-ac46-1a8c0b91e0a1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.348224 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c205b-af9b-4efd-ac46-1a8c0b91e0a1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.348338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c205b-af9b-4efd-ac46-1a8c0b91e0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.367131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6d2c205b-af9b-4efd-ac46-1a8c0b91e0a1 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:34.368395 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6d2c205b-af9b-4efd-ac46-1a8c0b91e0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:34.427638 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1404/2789] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:34.437779 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c0234-0564-42a8-8eba-540712e13aa4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.438283 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c0234-0564-42a8-8eba-540712e13aa4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.438387 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c0234-0564-42a8-8eba-540712e13aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.457360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-126c0234-0564-42a8-8eba-540712e13aa4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:34.458534 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-126c0234-0564-42a8-8eba-540712e13aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:34.466614 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1386/2790] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:06:34.471481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c205b-af9b-4efd-ac46-1a8c0b91e0a1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:34.481851 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c205b-af9b-4efd-ac46-1a8c0b91e0a1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:34.486873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:34.487629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.487916 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.488190 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.488939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:06:34.489245 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:34.490836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:34.502855 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1405/2791] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:06:34.507195 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c0234-0564-42a8-8eba-540712e13aa4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:34.517247 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c0234-0564-42a8-8eba-540712e13aa4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:34.521870 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbecb34-1a7c-4f51-8a8c-2cc889e3a33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:34.522517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbecb34-1a7c-4f51-8a8c-2cc889e3a33a admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.522806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbecb34-1a7c-4f51-8a8c-2cc889e3a33a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.523084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbecb34-1a7c-4f51-8a8c-2cc889e3a33a admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.523810 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbecb34-1a7c-4f51-8a8c-2cc889e3a33a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:34.525716 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbecb34-1a7c-4f51-8a8c-2cc889e3a33a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:34.611414 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1387/2792] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:06:34 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:34.615443 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:34.625549 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841f172-7468-413f-9562-828e15ca1326 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:34.648729 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:34.680064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c37d6-758e-4bd3-8880-68df61444b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:34.680759 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c37d6-758e-4bd3-8880-68df61444b45 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.681041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c37d6-758e-4bd3-8880-68df61444b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.681317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c37d6-758e-4bd3-8880-68df61444b45 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.681986 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c37d6-758e-4bd3-8880-68df61444b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:34.683917 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c37d6-758e-4bd3-8880-68df61444b45 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:34.798234 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1406/2793] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:06:34 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:34.802635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbecb34-1a7c-4f51-8a8c-2cc889e3a33a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:34.817725 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbecb34-1a7c-4f51-8a8c-2cc889e3a33a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:34.822620 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac393a0-b525-418e-b383-73f69595f9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:34.823318 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac393a0-b525-418e-b383-73f69595f9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.823623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac393a0-b525-418e-b383-73f69595f9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.823910 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac393a0-b525-418e-b383-73f69595f9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.824689 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac393a0-b525-418e-b383-73f69595f9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:34.827095 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac393a0-b525-418e-b383-73f69595f9a6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:34.839855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1388/2794] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:34.844154 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c37d6-758e-4bd3-8880-68df61444b45 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:34.854329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c37d6-758e-4bd3-8880-68df61444b45 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:34.859122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f195b-5dbb-4f35-92a0-c50e17b2c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:34.859845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f195b-5dbb-4f35-92a0-c50e17b2c9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.860134 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f195b-5dbb-4f35-92a0-c50e17b2c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.860415 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f195b-5dbb-4f35-92a0-c50e17b2c9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.861139 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f195b-5dbb-4f35-92a0-c50e17b2c9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:34.862790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f195b-5dbb-4f35-92a0-c50e17b2c9d7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:34.873944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1407/2795] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:34.878003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac393a0-b525-418e-b383-73f69595f9a6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:34.888125 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac393a0-b525-418e-b383-73f69595f9a6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:34.893919 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:34.894179 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:34.894496 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:34.894777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] PATH_INFO: `/v3/projects/b5530af77f8d4d319251e5aa27d8a536/users/4578eaca578d4064977c32a2e8eb3267/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:34.909209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5530af77f8d4d319251e5aa27d8a536, user_id=4578eaca578d4064977c32a2e8eb3267, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:34.911917 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:34.996706 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1389/2796] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:34 2022] PUT /identity/v3/projects/b5530af77f8d4d319251e5aa27d8a536/users/4578eaca578d4064977c32a2e8eb3267/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:35.002416 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba3e3-be87-4b7a-b54f-8dda71f492c0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:35.003045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba3e3-be87-4b7a-b54f-8dda71f492c0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:35.003475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba3e3-be87-4b7a-b54f-8dda71f492c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:35.022686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-be6ba3e3-be87-4b7a-b54f-8dda71f492c0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:35.023938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-be6ba3e3-be87-4b7a-b54f-8dda71f492c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:35.087767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1408/2797] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:35.097571 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:35.099277 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:35.155759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8534b5e-39e7-4bf5-b809-64200eb9db14 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:35.159343 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896666b-a29a-4a3b-90d3-4e9974d4fee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:35.159990 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896666b-a29a-4a3b-90d3-4e9974d4fee1 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:35.160275 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896666b-a29a-4a3b-90d3-4e9974d4fee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:35.160555 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896666b-a29a-4a3b-90d3-4e9974d4fee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:35.169471 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896666b-a29a-4a3b-90d3-4e9974d4fee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:35.171488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896666b-a29a-4a3b-90d3-4e9974d4fee1 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:35.176489 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:35.181136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1390/2798] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:41.556607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a61d5-b246-4994-b4ed-9c003bd89d24 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:41.557128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a61d5-b246-4994-b4ed-9c003bd89d24 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:41.557271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a61d5-b246-4994-b4ed-9c003bd89d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:41.591511 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-753a61d5-b246-4994-b4ed-9c003bd89d24 None None] MFA Rules not processed for user `4578eaca578d4064977c32a2e8eb3267`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:41.593667 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-753a61d5-b246-4994-b4ed-9c003bd89d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:41.665836 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1409/2799] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:06:41.669789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896666b-a29a-4a3b-90d3-4e9974d4fee1 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:41.680777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896666b-a29a-4a3b-90d3-4e9974d4fee1 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:41.685858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3a2a8-fdee-4625-921d-0c0987dfe5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:41.686609 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3a2a8-fdee-4625-921d-0c0987dfe5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:41.686899 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3a2a8-fdee-4625-921d-0c0987dfe5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:41.687183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3a2a8-fdee-4625-921d-0c0987dfe5d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:41.688224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3a2a8-fdee-4625-921d-0c0987dfe5d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:41.690154 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3a2a8-fdee-4625-921d-0c0987dfe5d5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:41.783878 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1391/2800] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:06:41 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:41.788578 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a61d5-b246-4994-b4ed-9c003bd89d24 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:41.800722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a61d5-b246-4994-b4ed-9c003bd89d24 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:41.834081 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:41.859188 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0f1b2-1d08-4811-b7fe-87aeb9213c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:41.859921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0f1b2-1d08-4811-b7fe-87aeb9213c52 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:41.860207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0f1b2-1d08-4811-b7fe-87aeb9213c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:41.860488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0f1b2-1d08-4811-b7fe-87aeb9213c52 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:41.861184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0f1b2-1d08-4811-b7fe-87aeb9213c52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:41.863199 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0f1b2-1d08-4811-b7fe-87aeb9213c52 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:41.958987 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1410/2801] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:06:41 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:41.963193 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3a2a8-fdee-4625-921d-0c0987dfe5d5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:41.974717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3a2a8-fdee-4625-921d-0c0987dfe5d5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:41.980033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c262a-6de3-48ce-9659-093694b72402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:41.980762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c262a-6de3-48ce-9659-093694b72402 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:41.980991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c262a-6de3-48ce-9659-093694b72402 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:41.981269 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c262a-6de3-48ce-9659-093694b72402 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:41.981994 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c262a-6de3-48ce-9659-093694b72402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:41.983715 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c262a-6de3-48ce-9659-093694b72402 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:41.994467 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1392/2802] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:41.998852 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0f1b2-1d08-4811-b7fe-87aeb9213c52 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:42.009166 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0f1b2-1d08-4811-b7fe-87aeb9213c52 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:42.014434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed6458-166c-48de-8894-95604467487d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:42.015341 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6458-166c-48de-8894-95604467487d admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:42.015633 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6458-166c-48de-8894-95604467487d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:42.015908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed6458-166c-48de-8894-95604467487d admin admin] PATH_INFO: `/v3/projects/200b109c72ac43b7a7f9eaa9c827d75d/users/fb2c8452a50345758f64e311e7195466/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:42.032497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed6458-166c-48de-8894-95604467487d admin admin] RBAC: Authorizing `identity:create_grant(project_id=200b109c72ac43b7a7f9eaa9c827d75d, user_id=fb2c8452a50345758f64e311e7195466, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:42.035192 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed6458-166c-48de-8894-95604467487d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:42.142337 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1411/2803] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:41 2022] PUT /identity/v3/projects/200b109c72ac43b7a7f9eaa9c827d75d/users/fb2c8452a50345758f64e311e7195466/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:42.145852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c262a-6de3-48ce-9659-093694b72402 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:42.156158 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c262a-6de3-48ce-9659-093694b72402 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:42.213452 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:42.214127 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:42.214418 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:42.214710 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:42.215462 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:42.217081 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:42.234696 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1393/2804] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:42.234816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed6458-166c-48de-8894-95604467487d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:42.242699 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed6458-166c-48de-8894-95604467487d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:42.248035 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a82e1c-cae1-42cd-8491-4aeae7e74255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:42.249449 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a82e1c-cae1-42cd-8491-4aeae7e74255 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:42.249735 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a82e1c-cae1-42cd-8491-4aeae7e74255 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:42.250011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a82e1c-cae1-42cd-8491-4aeae7e74255 admin admin] PATH_INFO: `/v3/projects/200b109c72ac43b7a7f9eaa9c827d75d/users/fb2c8452a50345758f64e311e7195466/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:42.252031 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a82e1c-cae1-42cd-8491-4aeae7e74255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200b109c72ac43b7a7f9eaa9c827d75d, user_id=fb2c8452a50345758f64e311e7195466, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:42.254773 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a82e1c-cae1-42cd-8491-4aeae7e74255 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:42.266324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1412/2805] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:42 2022] PUT /identity/v3/projects/200b109c72ac43b7a7f9eaa9c827d75d/users/fb2c8452a50345758f64e311e7195466/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:44.413693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:44.415773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:44.502914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a48ea-3c0c-4896-8564-2084f65878e3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:44.588094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:44.588778 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:44.589300 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:44.589405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:44.595264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:44.646514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:44.680094 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:44.681637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:44.695202 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1394/2806] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:49.012121 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496e625-a7a2-4c53-bd29-32b4c7123f7e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:49.012676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496e625-a7a2-4c53-bd29-32b4c7123f7e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:49.012676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496e625-a7a2-4c53-bd29-32b4c7123f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:49.051888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b496e625-a7a2-4c53-bd29-32b4c7123f7e None None] MFA Rules not processed for user `fb2c8452a50345758f64e311e7195466`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:49.053520 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b496e625-a7a2-4c53-bd29-32b4c7123f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:49.136207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1413/2807] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:49.152456 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496e625-a7a2-4c53-bd29-32b4c7123f7e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:49.152773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:49.155212 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b496e625-a7a2-4c53-bd29-32b4c7123f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:49.155645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:49.172869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca5253-3a92-4e7b-a065-829477dd0c49 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:49.187926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:49.187926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:49.187926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:49.187926 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] PATH_INFO: `/v3/users/1157b91a306540dd98a072b1ea4941b2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:49.188474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1157b91a306540dd98a072b1ea4941b2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:49.189839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:49.271032 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496e625-a7a2-4c53-bd29-32b4c7123f7e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:49.276874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6775419-33c2-4a8b-8408-586d7498d69c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:49.281945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6775419-33c2-4a8b-8408-586d7498d69c service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:49.281945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6775419-33c2-4a8b-8408-586d7498d69c service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:49.281945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6775419-33c2-4a8b-8408-586d7498d69c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:49.287486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:49.341605 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6775419-33c2-4a8b-8408-586d7498d69c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:49.344253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6775419-33c2-4a8b-8408-586d7498d69c service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:49.354101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1414/2808] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:06:49.429766 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1157b91a306540dd98a072b1ea4941b2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:49.442472 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1157b91a306540dd98a072b1ea4941b2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:49.443135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1157b91a306540dd98a072b1ea4941b2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:49.443795 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1157b91a306540dd98a072b1ea4941b2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:49.450827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c6315fef-c47c-4323-a0fd-1d1d2970fa27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1157b91a306540dd98a072b1ea4941b2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:49.539993 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1395/2809] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:06:49 2022] DELETE /identity/v3/users/1157b91a306540dd98a072b1ea4941b2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:49.799607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6775419-33c2-4a8b-8408-586d7498d69c service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:49.811404 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6775419-33c2-4a8b-8408-586d7498d69c service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:49.880804 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:49.882072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:49.882447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:49.882891 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] PATH_INFO: `/v3/projects/c35d3e93d36040bba1bbb873e61d239a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:49.884270 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35d3e93d36040bba1bbb873e61d239a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:49.886864 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:50.026870 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35d3e93d36040bba1bbb873e61d239a', 'request_id': 'req-0092f886-19fb-4156-91ab-4d79438784ae'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:50.032616 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35d3e93d36040bba1bbb873e61d239a', 'request_id': 'req-0092f886-19fb-4156-91ab-4d79438784ae'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:50.095257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35d3e93d36040bba1bbb873e61d239a', 'request_id': 'req-0092f886-19fb-4156-91ab-4d79438784ae'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:50.107942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] The token cache is being invalidate because project c35d3e93d36040bba1bbb873e61d239a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:06:50.108353 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:50.109006 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0092f886-19fb-4156-91ab-4d79438784ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:50.110545 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1415/2810] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:06:49 2022] DELETE /identity/v3/projects/c35d3e93d36040bba1bbb873e61d239a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:50.115653 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefdf7-83a5-422c-8cab-b1ad3d98cf94 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.115960 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefdf7-83a5-422c-8cab-b1ad3d98cf94 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.116251 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbefdf7-83a5-422c-8cab-b1ad3d98cf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.134551 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dfbefdf7-83a5-422c-8cab-b1ad3d98cf94 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:50.135806 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dfbefdf7-83a5-422c-8cab-b1ad3d98cf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:50.198495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1396/2811] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:50.205963 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139b58b-5a8e-4366-816b-65a02e83c2df None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.206329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139b58b-5a8e-4366-816b-65a02e83c2df None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.206686 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139b58b-5a8e-4366-816b-65a02e83c2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.227683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c139b58b-5a8e-4366-816b-65a02e83c2df None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:50.228884 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c139b58b-5a8e-4366-816b-65a02e83c2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:50.238128 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1416/2812] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:50.246859 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbefdf7-83a5-422c-8cab-b1ad3d98cf94 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:50.257707 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbefdf7-83a5-422c-8cab-b1ad3d98cf94 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:50.262559 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:50.263392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.263583 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.263894 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.264618 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:06:50.264915 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:50.266547 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:50.278834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1397/2813] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 16:06:50.283313 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139b58b-5a8e-4366-816b-65a02e83c2df None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:50.292794 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139b58b-5a8e-4366-816b-65a02e83c2df None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:50.298100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465885c4-2068-4167-b9ac-7047a0840440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:50.298746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465885c4-2068-4167-b9ac-7047a0840440 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.299033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465885c4-2068-4167-b9ac-7047a0840440 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.299338 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465885c4-2068-4167-b9ac-7047a0840440 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.300032 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465885c4-2068-4167-b9ac-7047a0840440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:50.301933 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465885c4-2068-4167-b9ac-7047a0840440 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:50.386796 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1417/2814] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:06:50 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:50.390907 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:50.405403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7659c-9f30-4a61-a14b-34bb6e26acfe admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:50.430010 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:50.462044 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0007be-3ec8-488d-85d2-2c490a9f1ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:50.462874 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0007be-3ec8-488d-85d2-2c490a9f1ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.463820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0007be-3ec8-488d-85d2-2c490a9f1ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.463820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0007be-3ec8-488d-85d2-2c490a9f1ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.464280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0007be-3ec8-488d-85d2-2c490a9f1ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:50.466244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0007be-3ec8-488d-85d2-2c490a9f1ab3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:50.552954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1398/2815] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:06:50 2022] POST /identity/v3/users => generated 303 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:50.557064 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465885c4-2068-4167-b9ac-7047a0840440 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:50.567812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465885c4-2068-4167-b9ac-7047a0840440 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:50.572452 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18606097-bed5-4362-9d5b-5e647e78a6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:50.573140 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606097-bed5-4362-9d5b-5e647e78a6dd admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.573551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606097-bed5-4362-9d5b-5e647e78a6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.574537 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606097-bed5-4362-9d5b-5e647e78a6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.575311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18606097-bed5-4362-9d5b-5e647e78a6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:50.578775 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18606097-bed5-4362-9d5b-5e647e78a6dd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:50.582554 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:50.588942 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1418/2816] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:50.593927 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0007be-3ec8-488d-85d2-2c490a9f1ab3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:50.603911 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0007be-3ec8-488d-85d2-2c490a9f1ab3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:50.609257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c0eab-5ae6-4331-9153-e98f9698284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:50.609988 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c0eab-5ae6-4331-9153-e98f9698284d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.610340 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c0eab-5ae6-4331-9153-e98f9698284d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.610681 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c0eab-5ae6-4331-9153-e98f9698284d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.611505 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c0eab-5ae6-4331-9153-e98f9698284d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:50.613168 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c0eab-5ae6-4331-9153-e98f9698284d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:50.623812 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1399/2817] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:50.628228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18606097-bed5-4362-9d5b-5e647e78a6dd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:50.638236 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18606097-bed5-4362-9d5b-5e647e78a6dd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:50.643227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:50.643981 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.644263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.644535 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] PATH_INFO: `/v3/projects/fbc4cc720d6f4c3c826163a2425217c0/users/e8a6276d3c0e44f58416202a83312bb1/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.659921 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc4cc720d6f4c3c826163a2425217c0, user_id=e8a6276d3c0e44f58416202a83312bb1, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:50.662551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:50.758646 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1419/2818] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:50 2022] PUT /identity/v3/projects/fbc4cc720d6f4c3c826163a2425217c0/users/e8a6276d3c0e44f58416202a83312bb1/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:06:50.763576 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cebba9-60e3-48df-840b-b92d1da0a3e9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.764503 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cebba9-60e3-48df-840b-b92d1da0a3e9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.764815 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cebba9-60e3-48df-840b-b92d1da0a3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.783739 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-88cebba9-60e3-48df-840b-b92d1da0a3e9 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:50.785590 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-88cebba9-60e3-48df-840b-b92d1da0a3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:50.853868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1400/2819] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:50.864056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:50.866413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:50.927992 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5e1b1-788e-4d93-95fa-a41edf5ba934 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:50.931767 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4de6e-5fbe-4f9b-9bc0-2063045f6f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:50.932428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4de6e-5fbe-4f9b-9bc0-2063045f6f46 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:50.932715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4de6e-5fbe-4f9b-9bc0-2063045f6f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:50.933107 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4de6e-5fbe-4f9b-9bc0-2063045f6f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:50.942559 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4de6e-5fbe-4f9b-9bc0-2063045f6f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:50.944043 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa4de6e-5fbe-4f9b-9bc0-2063045f6f46 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:50.955251 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1420/2820] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:52.549171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a614e-0c1b-45ef-b9bb-99171e41c054 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:52.549663 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a614e-0c1b-45ef-b9bb-99171e41c054 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:52.549782 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a614e-0c1b-45ef-b9bb-99171e41c054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:52.577761 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-eb1a614e-0c1b-45ef-b9bb-99171e41c054 None None] MFA Rules not processed for user `e8a6276d3c0e44f58416202a83312bb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:52.579004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-eb1a614e-0c1b-45ef-b9bb-99171e41c054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:52.654970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1401/2821] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:52 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:52.655714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4de6e-5fbe-4f9b-9bc0-2063045f6f46 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:52.665634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa4de6e-5fbe-4f9b-9bc0-2063045f6f46 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:52.671367 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ec0a0-0530-4f45-8699-6292fbe105d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:52.672095 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec0a0-0530-4f45-8699-6292fbe105d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:52.672373 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec0a0-0530-4f45-8699-6292fbe105d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:52.672648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec0a0-0530-4f45-8699-6292fbe105d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:52.673335 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ec0a0-0530-4f45-8699-6292fbe105d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:52.675497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ec0a0-0530-4f45-8699-6292fbe105d6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:52.754271 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1421/2822] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:06:52 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:06:52.755703 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a614e-0c1b-45ef-b9bb-99171e41c054 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:52.769172 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a614e-0c1b-45ef-b9bb-99171e41c054 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:52.810290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:52.847132 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ad857-9d3e-4942-b884-1cd47084e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:52.847326 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ad857-9d3e-4942-b884-1cd47084e264 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:52.847694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ad857-9d3e-4942-b884-1cd47084e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:52.848011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ad857-9d3e-4942-b884-1cd47084e264 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:52.848704 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ad857-9d3e-4942-b884-1cd47084e264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:52.850699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ad857-9d3e-4942-b884-1cd47084e264 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:52.977631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1402/2823] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:06:52 2022] POST /identity/v3/users => generated 308 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:52.980939 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ec0a0-0530-4f45-8699-6292fbe105d6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:52.992056 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ec0a0-0530-4f45-8699-6292fbe105d6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:53.011406 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213d14-7a8b-4c52-b76e-e006543ecb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:53.012088 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213d14-7a8b-4c52-b76e-e006543ecb23 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:53.012372 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213d14-7a8b-4c52-b76e-e006543ecb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:53.012642 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213d14-7a8b-4c52-b76e-e006543ecb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:53.014491 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3213d14-7a8b-4c52-b76e-e006543ecb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:53.016402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3213d14-7a8b-4c52-b76e-e006543ecb23 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:53.031488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1422/2824] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:53.034931 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ad857-9d3e-4942-b884-1cd47084e264 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:53.045901 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ad857-9d3e-4942-b884-1cd47084e264 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:53.051562 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9d90f-5daa-4908-9ecc-2137e5e412bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:53.054083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d90f-5daa-4908-9ecc-2137e5e412bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:53.054083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d90f-5daa-4908-9ecc-2137e5e412bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:53.054083 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9d90f-5daa-4908-9ecc-2137e5e412bb admin admin] PATH_INFO: `/v3/projects/2d73b608c2a04355b3862394ac8ad44e/users/38e8efedabe34a66a42c13d9359beb5f/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:53.073757 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9d90f-5daa-4908-9ecc-2137e5e412bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d73b608c2a04355b3862394ac8ad44e, user_id=38e8efedabe34a66a42c13d9359beb5f, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:53.077296 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9d90f-5daa-4908-9ecc-2137e5e412bb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:53.178633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1403/2825] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:53 2022] PUT /identity/v3/projects/2d73b608c2a04355b3862394ac8ad44e/users/38e8efedabe34a66a42c13d9359beb5f/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:53.181274 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213d14-7a8b-4c52-b76e-e006543ecb23 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:53.193975 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213d14-7a8b-4c52-b76e-e006543ecb23 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:53.241142 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9fbd6-c733-4b70-aac7-830280409b0b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:53.242374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9fbd6-c733-4b70-aac7-830280409b0b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:53.242744 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9fbd6-c733-4b70-aac7-830280409b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:53.265628 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a2e9fbd6-c733-4b70-aac7-830280409b0b None None] MFA Rules not processed for user `14fecc08cd364a48a8640d522577aa39`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:53.270206 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2e9fbd6-c733-4b70-aac7-830280409b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:53.275546 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6347-c987-44e7-ac51-a4eec256aa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:53.276765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6347-c987-44e7-ac51-a4eec256aa16 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:53.276765 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6347-c987-44e7-ac51-a4eec256aa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:53.277008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6347-c987-44e7-ac51-a4eec256aa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:53.277722 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff6347-c987-44e7-ac51-a4eec256aa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:53.280092 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff6347-c987-44e7-ac51-a4eec256aa16 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:53.290805 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1423/2826] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:53.294874 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6347-c987-44e7-ac51-a4eec256aa16 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:53.304883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6347-c987-44e7-ac51-a4eec256aa16 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:53.311442 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45e9d6-8d16-44c7-9dd3-f76cdb21359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:53.312468 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45e9d6-8d16-44c7-9dd3-f76cdb21359c admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:53.312716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45e9d6-8d16-44c7-9dd3-f76cdb21359c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:53.312994 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45e9d6-8d16-44c7-9dd3-f76cdb21359c admin admin] PATH_INFO: `/v3/projects/2d73b608c2a04355b3862394ac8ad44e/users/38e8efedabe34a66a42c13d9359beb5f/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:53.315797 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45e9d6-8d16-44c7-9dd3-f76cdb21359c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d73b608c2a04355b3862394ac8ad44e, user_id=38e8efedabe34a66a42c13d9359beb5f, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:53.318594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45e9d6-8d16-44c7-9dd3-f76cdb21359c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:53.330135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1424/2827] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:53 2022] PUT /identity/v3/projects/2d73b608c2a04355b3862394ac8ad44e/users/38e8efedabe34a66a42c13d9359beb5f/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:53.339262 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1404/2828] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:06:53.565024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c853e6-3433-46f2-a60c-f77c768dce1f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:53.565776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c853e6-3433-46f2-a60c-f77c768dce1f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:53.565776 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c853e6-3433-46f2-a60c-f77c768dce1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:53.602624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-38c853e6-3433-46f2-a60c-f77c768dce1f None None] MFA Rules not processed for user `4578eaca578d4064977c32a2e8eb3267`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:53.603909 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-38c853e6-3433-46f2-a60c-f77c768dce1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:53.647762 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:53.686355 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1425/2829] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:53.729404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9fbd6-c733-4b70-aac7-830280409b0b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:53.730868 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2e9fbd6-c733-4b70-aac7-830280409b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:53.820835 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9fbd6-c733-4b70-aac7-830280409b0b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:53.909604 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:53.910655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:53.911098 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:53.913176 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:53.918102 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:53.932192 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:53.933817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:53.972770 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1405/2830] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:06:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:06:54.988897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fb0bb-5f9e-424a-8be7-e73fdf4551da None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:54.991011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fb0bb-5f9e-424a-8be7-e73fdf4551da None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:54.991011 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fb0bb-5f9e-424a-8be7-e73fdf4551da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:55.016968 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-433fb0bb-5f9e-424a-8be7-e73fdf4551da None None] MFA Rules not processed for user `38e8efedabe34a66a42c13d9359beb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:55.018187 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-433fb0bb-5f9e-424a-8be7-e73fdf4551da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:55.164198 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1426/2831] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:56.771608 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:56.782672 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:56.842938 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38008b-6676-42a6-8b5f-3e3a8eb0eb9b service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:56.848360 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:56.849254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:56.849541 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:56.849819 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] PATH_INFO: `/v3/users/e8a6276d3c0e44f58416202a83312bb1` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:56.851043 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8a6276d3c0e44f58416202a83312bb1)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:56.853035 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:57.086747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8a6276d3c0e44f58416202a83312bb1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.107056 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8a6276d3c0e44f58416202a83312bb1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.107843 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8a6276d3c0e44f58416202a83312bb1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.108539 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8a6276d3c0e44f58416202a83312bb1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.115831 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8a6276d3c0e44f58416202a83312bb1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.205163 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1406/2832] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:06:56 2022] DELETE /identity/v3/users/e8a6276d3c0e44f58416202a83312bb1 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:57.516246 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fb0bb-5f9e-424a-8be7-e73fdf4551da None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:57.527746 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fb0bb-5f9e-424a-8be7-e73fdf4551da None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:57.588324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:57.589257 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:57.589538 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:57.589811 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] PATH_INFO: `/v3/projects/fbc4cc720d6f4c3c826163a2425217c0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:57.591133 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc4cc720d6f4c3c826163a2425217c0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:57.593896 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:57.755057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc4cc720d6f4c3c826163a2425217c0', 'request_id': 'req-415d5a7a-3adb-4cc0-8272-07aed3175f09'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.761072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc4cc720d6f4c3c826163a2425217c0', 'request_id': 'req-415d5a7a-3adb-4cc0-8272-07aed3175f09'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.764162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:57.866277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc4cc720d6f4c3c826163a2425217c0', 'request_id': 'req-415d5a7a-3adb-4cc0-8272-07aed3175f09'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.881263 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] The token cache is being invalidate because project fbc4cc720d6f4c3c826163a2425217c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:06:57.881774 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.882595 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:57.884790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1427/2833] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:06:57 2022] DELETE /identity/v3/projects/fbc4cc720d6f4c3c826163a2425217c0 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:57.889014 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:57.891106 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:57.907001 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f794d-f4b1-422a-a7ec-89d5ebcc4519 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:57.974131 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:57.975002 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:57.975327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:57.975675 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] PATH_INFO: `/v3/users/38e8efedabe34a66a42c13d9359beb5f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:57.976890 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e8efedabe34a66a42c13d9359beb5f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:57.978948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:58.114196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:06:58.234722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e8efedabe34a66a42c13d9359beb5f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:58.247902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e8efedabe34a66a42c13d9359beb5f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:58.251691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e8efedabe34a66a42c13d9359beb5f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:58.255382 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e8efedabe34a66a42c13d9359beb5f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:58.259861 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3b22744a-73d1-4f73-b43a-cb8ed64fdc67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e8efedabe34a66a42c13d9359beb5f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:58.450646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1407/2834] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:06:57 2022] DELETE /identity/v3/users/38e8efedabe34a66a42c13d9359beb5f => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:06:58.790028 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:58.791369 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:58.807744 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d5a7a-3adb-4cc0-8272-07aed3175f09 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:58.877142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:58.878019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:58.878308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:58.878591 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] PATH_INFO: `/v3/projects/2d73b608c2a04355b3862394ac8ad44e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:58.879843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d73b608c2a04355b3862394ac8ad44e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:58.882018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:59.066857 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d73b608c2a04355b3862394ac8ad44e', 'request_id': 'req-411736e9-25b3-4680-838a-06cd69c5227b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:59.072448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d73b608c2a04355b3862394ac8ad44e', 'request_id': 'req-411736e9-25b3-4680-838a-06cd69c5227b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:59.177155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d73b608c2a04355b3862394ac8ad44e', 'request_id': 'req-411736e9-25b3-4680-838a-06cd69c5227b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:59.189174 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] The token cache is being invalidate because project 2d73b608c2a04355b3862394ac8ad44e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:06:59.189475 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:59.190118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-411736e9-25b3-4680-838a-06cd69c5227b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:06:59.192067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1428/2835] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:06:58 2022] DELETE /identity/v3/projects/2d73b608c2a04355b3862394ac8ad44e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:06:59.197397 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81403af-8700-4279-83f6-97268b1fcba4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:59.197873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81403af-8700-4279-83f6-97268b1fcba4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:59.198156 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81403af-8700-4279-83f6-97268b1fcba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:59.220611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d81403af-8700-4279-83f6-97268b1fcba4 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:59.222306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d81403af-8700-4279-83f6-97268b1fcba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:59.289677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1408/2836] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:59.297777 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbeba3-4b10-42d6-ac3a-06885474f549 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:59.298069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbeba3-4b10-42d6-ac3a-06885474f549 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:59.298349 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbeba3-4b10-42d6-ac3a-06885474f549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:59.316888 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-63bbeba3-4b10-42d6-ac3a-06885474f549 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:06:59.318077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-63bbeba3-4b10-42d6-ac3a-06885474f549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:06:59.326142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1429/2837] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:06:59.331235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81403af-8700-4279-83f6-97268b1fcba4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:59.341480 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81403af-8700-4279-83f6-97268b1fcba4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:59.346495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:59.347151 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:59.347474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:59.347759 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:59.348482 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:06:59.348793 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:59.350410 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:59.365722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1409/2838] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:06:59.366873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbeba3-4b10-42d6-ac3a-06885474f549 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:59.383572 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbeba3-4b10-42d6-ac3a-06885474f549 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:59.391635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65c9b-f20e-4d42-b8df-d54227d29293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:59.392321 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65c9b-f20e-4d42-b8df-d54227d29293 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:59.392615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65c9b-f20e-4d42-b8df-d54227d29293 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:59.392889 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65c9b-f20e-4d42-b8df-d54227d29293 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:59.393598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65c9b-f20e-4d42-b8df-d54227d29293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:59.395571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65c9b-f20e-4d42-b8df-d54227d29293 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:59.498692 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1430/2839] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:06:59 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:59.503212 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:59.519611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28143ac9-ff54-402c-89f8-7c0b68fbab58 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:59.632422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27c786-abbd-4fa8-9e78-1ac750286643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:59.633187 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27c786-abbd-4fa8-9e78-1ac750286643 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:59.633479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27c786-abbd-4fa8-9e78-1ac750286643 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:59.633764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27c786-abbd-4fa8-9e78-1ac750286643 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:59.634631 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27c786-abbd-4fa8-9e78-1ac750286643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:59.636975 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27c786-abbd-4fa8-9e78-1ac750286643 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:59.753237 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1410/2840] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:06:59 2022] POST /identity/v3/users => generated 283 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:06:59.758184 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65c9b-f20e-4d42-b8df-d54227d29293 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:59.770816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65c9b-f20e-4d42-b8df-d54227d29293 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:59.778570 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01b5fe-6cbf-420a-8ca9-b2ef733bd3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:59.779253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01b5fe-6cbf-420a-8ca9-b2ef733bd3cb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:59.779587 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01b5fe-6cbf-420a-8ca9-b2ef733bd3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:59.779865 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01b5fe-6cbf-420a-8ca9-b2ef733bd3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:59.780883 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01b5fe-6cbf-420a-8ca9-b2ef733bd3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:59.782769 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01b5fe-6cbf-420a-8ca9-b2ef733bd3cb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:59.795374 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1431/2841] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:06:59.831273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27c786-abbd-4fa8-9e78-1ac750286643 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:59.862464 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27c786-abbd-4fa8-9e78-1ac750286643 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:59.870623 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb842e-529b-4d15-b4ea-282077d8f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:59.871479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb842e-529b-4d15-b4ea-282077d8f2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:59.871479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb842e-529b-4d15-b4ea-282077d8f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:59.873722 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb842e-529b-4d15-b4ea-282077d8f2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:59.875654 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb842e-529b-4d15-b4ea-282077d8f2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:59.879327 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb842e-529b-4d15-b4ea-282077d8f2d4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:06:59.896646 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1411/2842] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:06:59.901113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01b5fe-6cbf-420a-8ca9-b2ef733bd3cb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:06:59.911964 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01b5fe-6cbf-420a-8ca9-b2ef733bd3cb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:06:59.917072 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167dd8d-7334-429d-bff0-3fbf2b5184f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:06:59.919213 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167dd8d-7334-429d-bff0-3fbf2b5184f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:06:59.919598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167dd8d-7334-429d-bff0-3fbf2b5184f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:06:59.919947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167dd8d-7334-429d-bff0-3fbf2b5184f7 admin admin] PATH_INFO: `/v3/projects/f47c8e2501674dd0911dfa432ea9d9d2/users/51e2bd7b0011498b80bf6b8c3c331e68/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:06:59.941207 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167dd8d-7334-429d-bff0-3fbf2b5184f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f47c8e2501674dd0911dfa432ea9d9d2, user_id=51e2bd7b0011498b80bf6b8c3c331e68, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:06:59.944237 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167dd8d-7334-429d-bff0-3fbf2b5184f7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:00.050943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1432/2843] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:06:59 2022] PUT /identity/v3/projects/f47c8e2501674dd0911dfa432ea9d9d2/users/51e2bd7b0011498b80bf6b8c3c331e68/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:07:00.066425 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9812bd-de10-4502-8fd9-20d05e443aa7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:00.066776 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9812bd-de10-4502-8fd9-20d05e443aa7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:00.067161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9812bd-de10-4502-8fd9-20d05e443aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:00.098245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:00.105864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5f9812bd-de10-4502-8fd9-20d05e443aa7 None None] MFA Rules not processed for user `51e2bd7b0011498b80bf6b8c3c331e68`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:00.109225 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5f9812bd-de10-4502-8fd9-20d05e443aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:00.186622 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1412/2844] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:00.191721 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167dd8d-7334-429d-bff0-3fbf2b5184f7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:00.211068 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:00.256250 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167dd8d-7334-429d-bff0-3fbf2b5184f7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:00.261447 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33cc77-b256-4fb1-9ae3-2a536b406709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:00.262098 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33cc77-b256-4fb1-9ae3-2a536b406709 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:00.262381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33cc77-b256-4fb1-9ae3-2a536b406709 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:00.262653 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33cc77-b256-4fb1-9ae3-2a536b406709 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:00.263352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33cc77-b256-4fb1-9ae3-2a536b406709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:00.265966 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33cc77-b256-4fb1-9ae3-2a536b406709 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:00.387629 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1433/2845] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:07:00 2022] POST /identity/v3/projects => generated 363 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:07:00.391834 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9812bd-de10-4502-8fd9-20d05e443aa7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:00.403233 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9812bd-de10-4502-8fd9-20d05e443aa7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:00.462956 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137df88e-e3a8-4261-8b41-d0495dad16fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:00.463660 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df88e-e3a8-4261-8b41-d0495dad16fc admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:00.463947 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df88e-e3a8-4261-8b41-d0495dad16fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:00.464224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df88e-e3a8-4261-8b41-d0495dad16fc admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:00.464920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137df88e-e3a8-4261-8b41-d0495dad16fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:00.466842 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137df88e-e3a8-4261-8b41-d0495dad16fc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:00.589734 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1413/2846] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:07:00 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:07:00.591023 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33cc77-b256-4fb1-9ae3-2a536b406709 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:00.601669 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33cc77-b256-4fb1-9ae3-2a536b406709 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:00.606664 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00980d86-42bc-4d4e-8819-ce5fd6b2dbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:00.607347 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00980d86-42bc-4d4e-8819-ce5fd6b2dbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:00.607634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00980d86-42bc-4d4e-8819-ce5fd6b2dbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:00.607914 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00980d86-42bc-4d4e-8819-ce5fd6b2dbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:00.609117 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00980d86-42bc-4d4e-8819-ce5fd6b2dbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:00.610708 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00980d86-42bc-4d4e-8819-ce5fd6b2dbd3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:00.626700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1434/2847] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:07:00.630789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137df88e-e3a8-4261-8b41-d0495dad16fc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:00.641537 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137df88e-e3a8-4261-8b41-d0495dad16fc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:00.646243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2b60f-3f27-4d91-bcfd-e9c66ab6474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:00.647134 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2b60f-3f27-4d91-bcfd-e9c66ab6474b admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:00.647456 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2b60f-3f27-4d91-bcfd-e9c66ab6474b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:00.647743 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2b60f-3f27-4d91-bcfd-e9c66ab6474b admin admin] PATH_INFO: `/v3/projects/2d17fcde05264279ae9b734996797272/users/8f899fcd60694f1faf28633dba89464a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:00.663517 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2b60f-3f27-4d91-bcfd-e9c66ab6474b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d17fcde05264279ae9b734996797272, user_id=8f899fcd60694f1faf28633dba89464a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:00.669817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2b60f-3f27-4d91-bcfd-e9c66ab6474b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:00.795962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1414/2848] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:07:00 2022] PUT /identity/v3/projects/2d17fcde05264279ae9b734996797272/users/8f899fcd60694f1faf28633dba89464a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:07:00.802286 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00980d86-42bc-4d4e-8819-ce5fd6b2dbd3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:00.819247 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00980d86-42bc-4d4e-8819-ce5fd6b2dbd3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:00.886136 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8d82b-e7d7-420f-8c37-8c5cc3f021cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:00.887329 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8d82b-e7d7-420f-8c37-8c5cc3f021cb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:00.887803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8d82b-e7d7-420f-8c37-8c5cc3f021cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:00.888240 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8d82b-e7d7-420f-8c37-8c5cc3f021cb admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:00.889162 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8d82b-e7d7-420f-8c37-8c5cc3f021cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:00.891795 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8d82b-e7d7-420f-8c37-8c5cc3f021cb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:00.902364 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1435/2849] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:07:00.906731 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2b60f-3f27-4d91-bcfd-e9c66ab6474b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:00.917962 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2b60f-3f27-4d91-bcfd-e9c66ab6474b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:00.925264 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:00.927446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:00.927729 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:00.928011 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] PATH_INFO: `/v3/projects/2d17fcde05264279ae9b734996797272/users/8f899fcd60694f1faf28633dba89464a/roles/9813667366e34ffeba8eccc9db7c90eb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:00.930133 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d17fcde05264279ae9b734996797272, user_id=8f899fcd60694f1faf28633dba89464a, role_id=9813667366e34ffeba8eccc9db7c90eb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:00.932857 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:00.945298 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1415/2850] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:07:00 2022] PUT /identity/v3/projects/2d17fcde05264279ae9b734996797272/users/8f899fcd60694f1faf28633dba89464a/roles/9813667366e34ffeba8eccc9db7c90eb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:07:00.956615 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419f81d-949f-45c9-8e7f-f73ee6ff7879 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:00.956979 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419f81d-949f-45c9-8e7f-f73ee6ff7879 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:00.957267 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419f81d-949f-45c9-8e7f-f73ee6ff7879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:00.984772 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2419f81d-949f-45c9-8e7f-f73ee6ff7879 None None] MFA Rules not processed for user `8f899fcd60694f1faf28633dba89464a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:00.986008 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2419f81d-949f-45c9-8e7f-f73ee6ff7879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:01.066471 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1436/2851] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:01.081101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:01.083107 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:01.149550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd0223-e22a-4bc6-b4c7-c091e9628fa8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:01.153244 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9276f-d476-4018-a7c8-917723301092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:01.153886 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9276f-d476-4018-a7c8-917723301092 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:01.154173 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9276f-d476-4018-a7c8-917723301092 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:01.154479 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9276f-d476-4018-a7c8-917723301092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:01.163362 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:01.165802 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9276f-d476-4018-a7c8-917723301092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:01.167393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9276f-d476-4018-a7c8-917723301092 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:01.178012 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1416/2852] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:01 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:01.388304 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419f81d-949f-45c9-8e7f-f73ee6ff7879 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:01.398352 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419f81d-949f-45c9-8e7f-f73ee6ff7879 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:01.402127 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d7f22-0890-4da9-b061-ebd7e702c172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:01.402770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7f22-0890-4da9-b061-ebd7e702c172 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:01.403051 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7f22-0890-4da9-b061-ebd7e702c172 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:01.403358 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d7f22-0890-4da9-b061-ebd7e702c172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:01.414402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:01.464982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d7f22-0890-4da9-b061-ebd7e702c172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:01.466897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d7f22-0890-4da9-b061-ebd7e702c172 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:01.477209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1437/2853] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:05.172497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9276f-d476-4018-a7c8-917723301092 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:05.250848 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9276f-d476-4018-a7c8-917723301092 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:05.256344 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:05.257242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:05.257531 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:05.257805 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] PATH_INFO: `/v3/users/51e2bd7b0011498b80bf6b8c3c331e68` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:05.258989 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e2bd7b0011498b80bf6b8c3c331e68)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:05.260951 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:05.444991 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e2bd7b0011498b80bf6b8c3c331e68'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:05.457169 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e2bd7b0011498b80bf6b8c3c331e68'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:05.459485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e2bd7b0011498b80bf6b8c3c331e68'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:05.459485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e2bd7b0011498b80bf6b8c3c331e68'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:05.465669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e2bd7b0011498b80bf6b8c3c331e68'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:05.557033 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1417/2854] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:07:05 2022] DELETE /identity/v3/users/51e2bd7b0011498b80bf6b8c3c331e68 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:07:05.561624 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1cb4b-6c0a-457e-906c-ca59bbcc55d0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:05.562122 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1cb4b-6c0a-457e-906c-ca59bbcc55d0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:05.562360 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1cb4b-6c0a-457e-906c-ca59bbcc55d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:05.578944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-27a1cb4b-6c0a-457e-906c-ca59bbcc55d0 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:05.580164 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-27a1cb4b-6c0a-457e-906c-ca59bbcc55d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:05.637413 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1438/2855] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:05.647802 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:05.649284 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:05.706238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6d68-cf40-4a45-be73-15ffc3afd328 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:05.710005 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aee01-04ed-4b5d-b60b-ad338b96bdf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:05.710650 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aee01-04ed-4b5d-b60b-ad338b96bdf0 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:05.710939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aee01-04ed-4b5d-b60b-ad338b96bdf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:05.711220 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aee01-04ed-4b5d-b60b-ad338b96bdf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:05.721135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495aee01-04ed-4b5d-b60b-ad338b96bdf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:05.722582 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495aee01-04ed-4b5d-b60b-ad338b96bdf0 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:05.733727 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1418/2856] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:06.160770 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1cb4b-6c0a-457e-906c-ca59bbcc55d0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:06.162340 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-27a1cb4b-6c0a-457e-906c-ca59bbcc55d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:06.175768 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1cb4b-6c0a-457e-906c-ca59bbcc55d0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:06.180948 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:06.182000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:06.182000 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:06.182467 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] PATH_INFO: `/v3/projects/f47c8e2501674dd0911dfa432ea9d9d2` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:06.183311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f47c8e2501674dd0911dfa432ea9d9d2)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:06.185504 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:06.308220 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f47c8e2501674dd0911dfa432ea9d9d2', 'request_id': 'req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.316897 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f47c8e2501674dd0911dfa432ea9d9d2', 'request_id': 'req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.385077 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f47c8e2501674dd0911dfa432ea9d9d2', 'request_id': 'req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.397401 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] The token cache is being invalidate because project f47c8e2501674dd0911dfa432ea9d9d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:07:06.397690 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.398324 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.399908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1439/2857] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:07:06 2022] DELETE /identity/v3/projects/f47c8e2501674dd0911dfa432ea9d9d2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:07:06.403903 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aee01-04ed-4b5d-b60b-ad338b96bdf0 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:06.418764 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aee01-04ed-4b5d-b60b-ad338b96bdf0 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:06.432071 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:06.477405 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:06.478280 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:06.478568 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:06.478852 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] PATH_INFO: `/v3/users/8f899fcd60694f1faf28633dba89464a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:06.480069 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f899fcd60694f1faf28633dba89464a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:06.482021 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:06.732377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f899fcd60694f1faf28633dba89464a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.747775 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f899fcd60694f1faf28633dba89464a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.750864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f899fcd60694f1faf28633dba89464a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.750864 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f899fcd60694f1faf28633dba89464a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.763242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-52429d03-3179-48e4-82e1-5a907d527b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f899fcd60694f1faf28633dba89464a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:06.832198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1419/2858] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:07:06 2022] DELETE /identity/v3/users/8f899fcd60694f1faf28633dba89464a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:07:07.325094 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:07.326575 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:07.343015 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25da47-2c53-429a-b4d0-e82f3f33fa9a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:07.379323 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:07.412203 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:07.413078 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:07.413362 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:07.413641 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] PATH_INFO: `/v3/projects/2d17fcde05264279ae9b734996797272` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:07.414967 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d17fcde05264279ae9b734996797272)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:07.417017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:07.506928 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d17fcde05264279ae9b734996797272', 'request_id': 'req-0aa5aedf-30fc-42ff-a2e4-718018d95def'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:07.512863 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d17fcde05264279ae9b734996797272', 'request_id': 'req-0aa5aedf-30fc-42ff-a2e4-718018d95def'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:07.567656 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d17fcde05264279ae9b734996797272', 'request_id': 'req-0aa5aedf-30fc-42ff-a2e4-718018d95def'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:07.579742 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] The token cache is being invalidate because project 2d17fcde05264279ae9b734996797272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:07:07.579971 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:07.580604 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0aa5aedf-30fc-42ff-a2e4-718018d95def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:07.582138 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1440/2859] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:07:07 2022] DELETE /identity/v3/projects/2d17fcde05264279ae9b734996797272 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:07:07.589165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8d17c-cd4a-4b9b-b862-2593efe27188 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:07.589655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8d17c-cd4a-4b9b-b862-2593efe27188 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:07.589767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8d17c-cd4a-4b9b-b862-2593efe27188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:07.607182 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3be8d17c-cd4a-4b9b-b862-2593efe27188 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:07.608416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3be8d17c-cd4a-4b9b-b862-2593efe27188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:07.666970 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1420/2860] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:07.674812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1815b8-2e2f-4ad0-8503-36c979a7ba97 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:07.675101 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1815b8-2e2f-4ad0-8503-36c979a7ba97 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:07.675412 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1815b8-2e2f-4ad0-8503-36c979a7ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:07.693644 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6f1815b8-2e2f-4ad0-8503-36c979a7ba97 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:07.694840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6f1815b8-2e2f-4ad0-8503-36c979a7ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:07.703202 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1441/2861] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:07.707256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8d17c-cd4a-4b9b-b862-2593efe27188 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:07.716982 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8d17c-cd4a-4b9b-b862-2593efe27188 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:07.721880 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:07.722530 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:07.722826 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:07.723109 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:07.723875 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:07:07.724191 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:07.725807 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:07.737306 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1421/2862] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:07:07.741294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1815b8-2e2f-4ad0-8503-36c979a7ba97 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:07.750873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1815b8-2e2f-4ad0-8503-36c979a7ba97 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:07.755512 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230351a-eea0-4552-be42-ea085040111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:07.756157 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230351a-eea0-4552-be42-ea085040111c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:07.756444 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230351a-eea0-4552-be42-ea085040111c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:07.756723 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230351a-eea0-4552-be42-ea085040111c admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:07.757417 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230351a-eea0-4552-be42-ea085040111c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:07.759327 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230351a-eea0-4552-be42-ea085040111c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:07.844319 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1442/2863] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:07:07 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:07:07.848446 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:07.859573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59eb60f-6c67-4986-8692-199a524c5c3f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:07.881610 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:07.919215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566f066-1357-428f-8dc5-e7d16924ea47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:07.920786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566f066-1357-428f-8dc5-e7d16924ea47 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:07.920786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566f066-1357-428f-8dc5-e7d16924ea47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:07.920786 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b566f066-1357-428f-8dc5-e7d16924ea47 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:07.921809 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566f066-1357-428f-8dc5-e7d16924ea47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:07.923845 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b566f066-1357-428f-8dc5-e7d16924ea47 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:08.014377 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1422/2864] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:07:07 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:07:08.018705 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230351a-eea0-4552-be42-ea085040111c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:08.028929 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230351a-eea0-4552-be42-ea085040111c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:08.033507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539940f4-3d1e-4a2f-bd34-37db5dd69f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:08.034155 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539940f4-3d1e-4a2f-bd34-37db5dd69f5f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:08.034441 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539940f4-3d1e-4a2f-bd34-37db5dd69f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:08.034720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539940f4-3d1e-4a2f-bd34-37db5dd69f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:08.035462 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539940f4-3d1e-4a2f-bd34-37db5dd69f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:08.037049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539940f4-3d1e-4a2f-bd34-37db5dd69f5f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:08.047801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1443/2865] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:07:08.051742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566f066-1357-428f-8dc5-e7d16924ea47 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:08.062393 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b566f066-1357-428f-8dc5-e7d16924ea47 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:08.067119 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deac7d-7be5-4b4b-be5c-9aad603d8964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:08.067808 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deac7d-7be5-4b4b-be5c-9aad603d8964 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:08.068101 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deac7d-7be5-4b4b-be5c-9aad603d8964 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:08.068409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deac7d-7be5-4b4b-be5c-9aad603d8964 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:08.069129 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deac7d-7be5-4b4b-be5c-9aad603d8964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:08.070737 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deac7d-7be5-4b4b-be5c-9aad603d8964 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:08.080856 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1423/2866] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:07:08.084867 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539940f4-3d1e-4a2f-bd34-37db5dd69f5f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:08.094977 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539940f4-3d1e-4a2f-bd34-37db5dd69f5f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:08.099481 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:08.100350 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:08.100635 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:08.100908 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] PATH_INFO: `/v3/projects/75726b42e3804ca2a310f015d2f0c2db/users/533240c7541641e3b3a4958d97d4845f/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:08.115630 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=75726b42e3804ca2a310f015d2f0c2db, user_id=533240c7541641e3b3a4958d97d4845f, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:08.118260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:08.185982 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1444/2867] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:07:08 2022] PUT /identity/v3/projects/75726b42e3804ca2a310f015d2f0c2db/users/533240c7541641e3b3a4958d97d4845f/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:07:08.197777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbbec2-e89b-4fec-be6d-1948dc667088 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:08.198073 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbbec2-e89b-4fec-be6d-1948dc667088 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:08.198356 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbbec2-e89b-4fec-be6d-1948dc667088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:08.225164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7edbbec2-e89b-4fec-be6d-1948dc667088 None None] MFA Rules not processed for user `533240c7541641e3b3a4958d97d4845f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:08.226379 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7edbbec2-e89b-4fec-be6d-1948dc667088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:08.290944 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1424/2868] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:08.302151 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:08.303965 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:08.360703 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1b12e-d834-4e73-a56e-7a3a128a54cf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:08.364283 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fbcc9-0d46-4d01-8d2a-83b2d464fd9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:08.364931 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbcc9-0d46-4d01-8d2a-83b2d464fd9b service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:08.365216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbcc9-0d46-4d01-8d2a-83b2d464fd9b service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:08.365498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbcc9-0d46-4d01-8d2a-83b2d464fd9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:08.373725 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fbcc9-0d46-4d01-8d2a-83b2d464fd9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:08.375216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fbcc9-0d46-4d01-8d2a-83b2d464fd9b service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:08.384293 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1445/2869] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:13.492436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbbec2-e89b-4fec-be6d-1948dc667088 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:13.554149 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbbec2-e89b-4fec-be6d-1948dc667088 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:13.558692 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:13.559662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:13.559954 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:13.560238 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] PATH_INFO: `/v3/users/533240c7541641e3b3a4958d97d4845f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:13.561451 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=533240c7541641e3b3a4958d97d4845f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:13.564004 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:13.785912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:13.795992 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '533240c7541641e3b3a4958d97d4845f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:13.809057 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '533240c7541641e3b3a4958d97d4845f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:13.809758 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '533240c7541641e3b3a4958d97d4845f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:13.810373 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '533240c7541641e3b3a4958d97d4845f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:13.817837 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '533240c7541641e3b3a4958d97d4845f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:13.884082 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1425/2870] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:07:13 2022] DELETE /identity/v3/users/533240c7541641e3b3a4958d97d4845f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:07:13.888579 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6edf2-69a1-4917-aa1f-58960a507fbc None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:13.890432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6edf2-69a1-4917-aa1f-58960a507fbc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:13.890432 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6edf2-69a1-4917-aa1f-58960a507fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:13.895639 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:13.908046 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-eed6edf2-69a1-4917-aa1f-58960a507fbc None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:13.909260 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-eed6edf2-69a1-4917-aa1f-58960a507fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:13.973142 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1446/2871] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:13.983687 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:13.985242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:14.045361 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6e463-6ebb-44b5-8bf3-35216efb9e0f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:14.049103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b79f3-3542-4b4e-9595-0b177c733bba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:14.049756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b79f3-3542-4b4e-9595-0b177c733bba service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:14.050065 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b79f3-3542-4b4e-9595-0b177c733bba service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:14.050332 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b79f3-3542-4b4e-9595-0b177c733bba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:14.059407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b79f3-3542-4b4e-9595-0b177c733bba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:14.060888 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b79f3-3542-4b4e-9595-0b177c733bba service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:14.070474 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1426/2872] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:14.474057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6edf2-69a1-4917-aa1f-58960a507fbc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:14.476173 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-eed6edf2-69a1-4917-aa1f-58960a507fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:14.487386 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6edf2-69a1-4917-aa1f-58960a507fbc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:14.491953 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:14.492840 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:14.493120 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:14.493407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] PATH_INFO: `/v3/projects/75726b42e3804ca2a310f015d2f0c2db` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:14.494564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75726b42e3804ca2a310f015d2f0c2db)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:14.496497 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:14.620827 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75726b42e3804ca2a310f015d2f0c2db', 'request_id': 'req-163155c3-7dbd-497f-988a-8a407e995084'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:14.626751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75726b42e3804ca2a310f015d2f0c2db', 'request_id': 'req-163155c3-7dbd-497f-988a-8a407e995084'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:14.688756 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75726b42e3804ca2a310f015d2f0c2db', 'request_id': 'req-163155c3-7dbd-497f-988a-8a407e995084'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:14.700167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] The token cache is being invalidate because project 75726b42e3804ca2a310f015d2f0c2db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:07:14.700460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:14.701099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-163155c3-7dbd-497f-988a-8a407e995084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:07:14.702636 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1447/2873] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:07:14 2022] DELETE /identity/v3/projects/75726b42e3804ca2a310f015d2f0c2db => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:07:14.707688 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f527c2-c227-4ac0-bfd9-067d5ac0b0f5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:14.708152 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f527c2-c227-4ac0-bfd9-067d5ac0b0f5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:14.708250 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f527c2-c227-4ac0-bfd9-067d5ac0b0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:14.726380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f7f527c2-c227-4ac0-bfd9-067d5ac0b0f5 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:14.727611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f7f527c2-c227-4ac0-bfd9-067d5ac0b0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:14.788242 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1427/2874] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:14.795683 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115074-133c-45c9-ba9f-3a97a8509155 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:14.796459 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115074-133c-45c9-ba9f-3a97a8509155 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:14.796749 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc115074-133c-45c9-ba9f-3a97a8509155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:14.815553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fc115074-133c-45c9-ba9f-3a97a8509155 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:14.817488 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fc115074-133c-45c9-ba9f-3a97a8509155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:14.824726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1448/2875] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 16:07:14.829256 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f527c2-c227-4ac0-bfd9-067d5ac0b0f5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:14.839204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f527c2-c227-4ac0-bfd9-067d5ac0b0f5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:14.844514 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:14.845183 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:14.845472 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:14.845754 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:14.846491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 16:07:14.846803 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:14.848459 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:14.861550 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1428/2876] 10.222.0.98 () {58 vars in 1152 bytes} [Fri Jul 15 16:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 16:07:14.865807 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc115074-133c-45c9-ba9f-3a97a8509155 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:14.876123 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc115074-133c-45c9-ba9f-3a97a8509155 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:14.881100 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:14.881947 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:14.882305 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:14.882685 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:14.883486 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:14.885460 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:14.993294 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1449/2877] 10.222.0.98 () {60 vars in 1152 bytes} [Fri Jul 15 16:07:14 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 16:07:14.997403 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:15.008691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b91330-4ff5-4639-a719-2b9b43f8fc81 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:15.049914 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:15.066637 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9410f-6ccf-48e1-809f-65de89a398df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:15.067350 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9410f-6ccf-48e1-809f-65de89a398df admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:15.067643 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9410f-6ccf-48e1-809f-65de89a398df admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:15.067959 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9410f-6ccf-48e1-809f-65de89a398df admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:15.068665 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9410f-6ccf-48e1-809f-65de89a398df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:15.070705 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9410f-6ccf-48e1-809f-65de89a398df admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:15.139135 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:15.140915 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:15.145817 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1429/2878] 10.222.0.98 () {60 vars in 1143 bytes} [Fri Jul 15 16:07:14 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 16:07:15.150755 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9410f-6ccf-48e1-809f-65de89a398df admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:15.161669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9410f-6ccf-48e1-809f-65de89a398df admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:15.166501 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21d850-b422-4745-a75a-ef24b5d67392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:15.167160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21d850-b422-4745-a75a-ef24b5d67392 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:15.167488 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21d850-b422-4745-a75a-ef24b5d67392 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:15.167773 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21d850-b422-4745-a75a-ef24b5d67392 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:15.168492 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21d850-b422-4745-a75a-ef24b5d67392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:15.170123 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21d850-b422-4745-a75a-ef24b5d67392 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:15.181406 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1430/2879] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 16:07:15.185367 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21d850-b422-4745-a75a-ef24b5d67392 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:15.194983 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21d850-b422-4745-a75a-ef24b5d67392 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:15.199756 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f7796-ddd4-4a8b-9206-5670cbf5a0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:15.200402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f7796-ddd4-4a8b-9206-5670cbf5a0cf admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:15.200694 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f7796-ddd4-4a8b-9206-5670cbf5a0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:15.200979 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f7796-ddd4-4a8b-9206-5670cbf5a0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:15.201777 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f7796-ddd4-4a8b-9206-5670cbf5a0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:15.202602 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cad01a-3c7a-42d3-954c-6e5cdafb8dd8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:15.203485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f7796-ddd4-4a8b-9206-5670cbf5a0cf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:15.206222 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f409db-cf19-480d-bdd2-0092907edfd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:15.206885 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f409db-cf19-480d-bdd2-0092907edfd1 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:15.207167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f409db-cf19-480d-bdd2-0092907edfd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:15.207477 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f409db-cf19-480d-bdd2-0092907edfd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:15.212331 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-93f409db-cf19-480d-bdd2-0092907edfd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:15.214076 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1431/2880] 10.222.0.98 () {58 vars in 1121 bytes} [Fri Jul 15 16:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 16:07:15.218602 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f7796-ddd4-4a8b-9206-5670cbf5a0cf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:15.222571 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:15.228677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f7796-ddd4-4a8b-9206-5670cbf5a0cf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:15.234297 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:15.235383 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:15.235747 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:15.236028 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] PATH_INFO: `/v3/projects/79622b5f74f44e4c9b7848cc490eab20/users/c35729b6d0e94098ab8e910dbf79ab7e/roles/0892f8baef1c4b61b13187ba945d2ad5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:15.251939 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79622b5f74f44e4c9b7848cc490eab20, user_id=c35729b6d0e94098ab8e910dbf79ab7e, role_id=0892f8baef1c4b61b13187ba945d2ad5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:15.254651 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:15.278067 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f409db-cf19-480d-bdd2-0092907edfd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:15.279553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f409db-cf19-480d-bdd2-0092907edfd1 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:15.289943 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1450/2881] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 16:07:15.329402 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1432/2882] 10.222.0.98 () {60 vars in 1482 bytes} [Fri Jul 15 16:07:15 2022] PUT /identity/v3/projects/79622b5f74f44e4c9b7848cc490eab20/users/c35729b6d0e94098ab8e910dbf79ab7e/roles/0892f8baef1c4b61b13187ba945d2ad5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 16:07:15.344253 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722048-e742-40c1-ba90-fff02a860365 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:15.344552 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722048-e742-40c1-ba90-fff02a860365 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:15.344876 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722048-e742-40c1-ba90-fff02a860365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:15.377226 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a7722048-e742-40c1-ba90-fff02a860365 None None] MFA Rules not processed for user `c35729b6d0e94098ab8e910dbf79ab7e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:07:15.378454 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a7722048-e742-40c1-ba90-fff02a860365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:15.444665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1451/2883] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:07:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:07:15.456849 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:15.458621 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:15.539558 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee87e3-75a2-4466-ab65-37f96be669d6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:15.543422 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52f18d0f5f634d84a99fc3e46963a8b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:15.544096 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:15.544380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:15.544656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:15.554371 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:15.555949 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:15.566635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1433/2884] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:17.478882 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7722048-e742-40c1-ba90-fff02a860365 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:17.484666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a7722048-e742-40c1-ba90-fff02a860365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:17.567715 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7722048-e742-40c1-ba90-fff02a860365 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:17.577810 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:17.578518 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:17.578803 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:17.579079 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:17.581021 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:17.594607 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:17.596242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:17.606984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1452/2885] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:20.700067 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:20.701506 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:20.772310 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca97acd-bfa1-4526-bcba-548b89feecb1 service glance] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:20.779450 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:20.780124 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:20.780416 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:20.780699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:20.782524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:20.794767 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:20.796920 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:20.801281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:20.807438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1434/2886] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:27.602317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:30.900442 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:07:39.059781 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:07:39.114093 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328f79e-1348-4825-b973-0bba0bcdecef service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:07:39.117938 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150168d1-fbc8-414a-b6e6-0a5f4438b747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:07:39.118586 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150168d1-fbc8-414a-b6e6-0a5f4438b747 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:07:39.118871 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150168d1-fbc8-414a-b6e6-0a5f4438b747 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:07:39.119156 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150168d1-fbc8-414a-b6e6-0a5f4438b747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:07:39.120952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-150168d1-fbc8-414a-b6e6-0a5f4438b747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:07:39.184200 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150168d1-fbc8-414a-b6e6-0a5f4438b747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:07:39.185652 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150168d1-fbc8-414a-b6e6-0a5f4438b747 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:07:39.195892 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1453/2887] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:07:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:07:49.191527 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=50 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:08:04.208054 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:08:04.270986 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febae450-09e5-4691-a8f9-f9ee2282b0de service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:08:04.275762 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:08:04.276635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:08:04.276925 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:08:04.277209 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] PATH_INFO: `/v3/users/c35729b6d0e94098ab8e910dbf79ab7e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:08:04.278438 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c35729b6d0e94098ab8e910dbf79ab7e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:08:04.280429 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:08:04.508312 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c35729b6d0e94098ab8e910dbf79ab7e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:04.520349 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c35729b6d0e94098ab8e910dbf79ab7e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:04.521030 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c35729b6d0e94098ab8e910dbf79ab7e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:04.521645 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c35729b6d0e94098ab8e910dbf79ab7e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:04.528486 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c35729b6d0e94098ab8e910dbf79ab7e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:04.598399 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1435/2888] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:08:04 2022] DELETE /identity/v3/users/c35729b6d0e94098ab8e910dbf79ab7e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:08:04.602887 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f7d06-70b3-415e-a184-3fe5f22244c7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:08:04.603425 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f7d06-70b3-415e-a184-3fe5f22244c7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:08:04.603543 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f7d06-70b3-415e-a184-3fe5f22244c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:08:04.620634 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e85f7d06-70b3-415e-a184-3fe5f22244c7 None None] MFA Rules not processed for user `fb07001f45bb40c8b8c84b7d67d1c304`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:08:04.621842 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e85f7d06-70b3-415e-a184-3fe5f22244c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:08:04.679587 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1454/2889] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:08:04.689086 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:08:04.690526 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:08:04.747656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f50c0b-9b26-418a-85ca-efc4312dd590 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:08:04.751222 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c86e1e-8b06-4ca8-8825-9e47d13ad73c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:08:04.752409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c86e1e-8b06-4ca8-8825-9e47d13ad73c service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:08:04.752409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c86e1e-8b06-4ca8-8825-9e47d13ad73c service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:08:04.752409 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c86e1e-8b06-4ca8-8825-9e47d13ad73c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:08:04.762161 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c86e1e-8b06-4ca8-8825-9e47d13ad73c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:08:04.763578 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c86e1e-8b06-4ca8-8825-9e47d13ad73c service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:08:04.773495 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1436/2890] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:08:05.187577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f7d06-70b3-415e-a184-3fe5f22244c7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:08:05.192498 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e85f7d06-70b3-415e-a184-3fe5f22244c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:08:05.204780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f7d06-70b3-415e-a184-3fe5f22244c7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:08:05.209557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:08:05.210423 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:08:05.210853 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:08:05.211227 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] PATH_INFO: `/v3/projects/79622b5f74f44e4c9b7848cc490eab20` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:08:05.212458 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79622b5f74f44e4c9b7848cc490eab20)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:08:05.214326 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:08:05.380466 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79622b5f74f44e4c9b7848cc490eab20', 'request_id': 'req-99db7fb1-eed1-492e-9fb0-309325c324f4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:05.387465 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79622b5f74f44e4c9b7848cc490eab20', 'request_id': 'req-99db7fb1-eed1-492e-9fb0-309325c324f4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:05.455131 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79622b5f74f44e4c9b7848cc490eab20', 'request_id': 'req-99db7fb1-eed1-492e-9fb0-309325c324f4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:05.466969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] The token cache is being invalidate because project 79622b5f74f44e4c9b7848cc490eab20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:08:05.467261 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:05.467935 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:08:05.473553 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1455/2891] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:08:05 2022] DELETE /identity/v3/projects/79622b5f74f44e4c9b7848cc490eab20 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:08:14.769068 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:08:15.463972 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:09:34.257010 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c86e1e-8b06-4ca8-8825-9e47d13ad73c service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:09:34.267546 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c86e1e-8b06-4ca8-8825-9e47d13ad73c service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:09:34.313633 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:09:34.314273 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:09:34.314556 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:09:34.314838 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:09:34.316662 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:34.325867 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:09:34.327320 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:09:34.335236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1437/2892] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:09:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:09:44.331611 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=26 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:09:49.113806 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:09:49.115216 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:49.127275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7fb1-eed1-492e-9fb0-309325c324f4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:09:49.131431 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da290-a670-4642-88a9-90f8d8c0363a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:09:49.132069 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da290-a670-4642-88a9-90f8d8c0363a service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:09:49.132354 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da290-a670-4642-88a9-90f8d8c0363a service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:09:49.132631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da290-a670-4642-88a9-90f8d8c0363a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:09:49.134357 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-149da290-a670-4642-88a9-90f8d8c0363a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:49.204596 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149da290-a670-4642-88a9-90f8d8c0363a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:09:49.205974 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149da290-a670-4642-88a9-90f8d8c0363a service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:09:49.216973 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1456/2893] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:09:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:09:50.394998 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:09:50.396443 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:50.406753 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5356af-6da5-4b10-8d9d-79f0a343e998 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:09:50.410401 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:09:50.411034 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:09:50.411353 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:09:50.411632 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:09:50.413386 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:50.427029 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:09:50.428491 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:09:50.439211 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1438/2894] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:09:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:09:53.815277 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f613bda-7d51-49e4-8fa9-342e3f37b284 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:09:53.816275 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f613bda-7d51-49e4-8fa9-342e3f37b284 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:09:53.816384 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f613bda-7d51-49e4-8fa9-342e3f37b284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:09:53.833665 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7f613bda-7d51-49e4-8fa9-342e3f37b284 None None] MFA Rules not processed for user `14fecc08cd364a48a8640d522577aa39`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:09:53.834866 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7f613bda-7d51-49e4-8fa9-342e3f37b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:53.887743 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1457/2895] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:09:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:09:54.191522 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:09:54.200669 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59888323-85bd-4c52-8c27-0f478afb72e7 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:09:54.204290 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:09:54.204934 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:09:54.205224 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:09:54.205507 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:09:54.207281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:54.216693 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:09:54.218126 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:09:54.226352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1439/2896] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:09:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:09:57.013873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f613bda-7d51-49e4-8fa9-342e3f37b284 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:09:57.015311 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7f613bda-7d51-49e4-8fa9-342e3f37b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:57.084945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f613bda-7d51-49e4-8fa9-342e3f37b284 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:09:57.090024 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:09:57.090676 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:09:57.090957 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:09:57.091230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:09:57.093144 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:09:57.106183 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:09:57.107809 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:09:57.118167 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1458/2897] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:09:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:10:03.971677 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:10:03.974048 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:10:03.993208 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858b128-b76c-438f-8200-c7ae73ea58a2 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:10:04.001691 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:10:04.002515 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:10:04.002876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:10:04.003254 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:10:04.005391 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:10:04.019869 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:10:04.021519 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:10:04.033690 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1440/2898] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:10:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:10:07.114448 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:10:13.451557 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:10:13.461564 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3721354-c58f-40cb-8522-11f843fae52b service placement] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:10:13.465254 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:10:13.465898 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:10:13.466185 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:10:13.466461 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:10:13.468265 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:10:13.481707 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:10:13.483150 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:10:13.492732 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1459/2899] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:10:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:10:13.817196 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:10:13.826656 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668dbacf-2e45-4beb-ba9b-8de3badaf4be service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:10:13.830234 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:10:13.830873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:10:13.831160 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:10:13.832003 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:10:13.833742 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:10:13.846839 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:10:13.848908 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:10:13.860215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1441/2900] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:10:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:10:14.340196 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:10:14.354859 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0543c-7451-4058-9afd-d5f103cbf768 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:10:14.362172 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:10:14.363402 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:10:14.363700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:10:14.363984 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:10:14.366108 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:10:14.383758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:10:14.383758 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:10:14.394215 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1460/2901] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:10:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:10:23.855204 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=24 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:10:24.389115 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=12 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:12:00.361472 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:00.399267 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d2dc0-c4a5-4efa-9bec-0bb25dd63da6 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:00.402943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:00.403624 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:00.403916 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:00.404199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:00.406045 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:00.486380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:00.487873 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:00.497372 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1442/2902] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:06.828598 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:06.854509 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38606a-33c8-4766-b0b3-ecf9d9d5be3a service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:06.860053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:06.860053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:06.860053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:06.860053 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:06.861654 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:06.954831 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:06.956211 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:06.975158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1461/2903] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:09.540820 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:09.543407 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:09.626049 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab50846-76c9-4333-b748-c548fc5425ef service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:09.630223 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68015ac9febe42ef917ca58fae7b1c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:09.630943 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:09.631437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:09.631718 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:09.633825 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:09.651524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:09.651524 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:09.658185 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1443/2904] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:10.626033 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:10.648850 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc3a1b-70a3-4f41-b8d3-c9b340500b3a service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:10.654343 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:10.655002 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:10.655308 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:10.655611 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:10.657480 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:10.684782 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:10.686381 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:10.698658 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1462/2905] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:14.397436 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:14.420292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff256d-9204-47fa-be8b-2cd601ff9fe2 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:14.420292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:14.420292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:14.420292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:14.420830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:14.420830 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:14.431827 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:14.433448 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:14.443520 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1444/2906] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:14.524392 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:14.526383 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:14.540428 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6483f3-1390-43d9-9323-b2bc9250e35a service placement] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:14.545073 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:14.545980 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:14.546228 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:14.546516 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] PATH_INFO: `/v3/users/4578eaca578d4064977c32a2e8eb3267` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:14.547733 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4578eaca578d4064977c32a2e8eb3267)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:14.549648 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:14.751645 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4578eaca578d4064977c32a2e8eb3267'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:14.763726 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4578eaca578d4064977c32a2e8eb3267'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:14.764410 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4578eaca578d4064977c32a2e8eb3267'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:14.765045 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4578eaca578d4064977c32a2e8eb3267'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:14.769530 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:12:14.772099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4578eaca578d4064977c32a2e8eb3267'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:14.881558 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1463/2907] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:12:14 2022] DELETE /identity/v3/users/4578eaca578d4064977c32a2e8eb3267 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:12:15.143078 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:15.154699 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db7391-3582-4d39-bcf6-722dcc341448 service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:15.173266 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:12:15.213246 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:15.214130 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:15.214741 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:15.215052 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] PATH_INFO: `/v3/projects/b5530af77f8d4d319251e5aa27d8a536` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:15.216281 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5530af77f8d4d319251e5aa27d8a536)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:15.218243 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:15.343164 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5530af77f8d4d319251e5aa27d8a536', 'request_id': 'req-8c262462-cd93-4984-8caa-e0ba173f2d2d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.349258 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5530af77f8d4d319251e5aa27d8a536', 'request_id': 'req-8c262462-cd93-4984-8caa-e0ba173f2d2d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.354658 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5530af77f8d4d319251e5aa27d8a536', 'request_id': 'req-8c262462-cd93-4984-8caa-e0ba173f2d2d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.423564 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] The token cache is being invalidate because project b5530af77f8d4d319251e5aa27d8a536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:12:15.424155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.424635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.426205 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1445/2908] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:12:15 2022] DELETE /identity/v3/projects/b5530af77f8d4d319251e5aa27d8a536 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:12:15.430780 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:15.432209 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:15.446310 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea36e1-2c4c-4c92-a1c8-fae52b957ea9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:15.502365 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:15.503242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:15.503551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:15.503830 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] PATH_INFO: `/v3/users/fb2c8452a50345758f64e311e7195466` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:15.505017 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2c8452a50345758f64e311e7195466)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:15.506926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:15.709843 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2c8452a50345758f64e311e7195466'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.722359 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2c8452a50345758f64e311e7195466'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.723099 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2c8452a50345758f64e311e7195466'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.723849 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2c8452a50345758f64e311e7195466'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.730944 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0642fdf5-8590-4f83-86c4-860072e565e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2c8452a50345758f64e311e7195466'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:15.842118 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1464/2909] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:12:15 2022] DELETE /identity/v3/users/fb2c8452a50345758f64e311e7195466 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:12:16.115485 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:16.129165 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:16.131933 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262462-cd93-4984-8caa-e0ba173f2d2d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:16.190912 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:16.191923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:16.192282 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:16.192569 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] PATH_INFO: `/v3/projects/200b109c72ac43b7a7f9eaa9c827d75d` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:16.193765 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200b109c72ac43b7a7f9eaa9c827d75d)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:16.195801 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:16.356591 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200b109c72ac43b7a7f9eaa9c827d75d', 'request_id': 'req-be254f23-a21b-47f6-9ac4-ce209d25f893'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:16.358749 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:12:16.363516 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200b109c72ac43b7a7f9eaa9c827d75d', 'request_id': 'req-be254f23-a21b-47f6-9ac4-ce209d25f893'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:16.370198 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200b109c72ac43b7a7f9eaa9c827d75d', 'request_id': 'req-be254f23-a21b-47f6-9ac4-ce209d25f893'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:16.466543 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] The token cache is being invalidate because project 200b109c72ac43b7a7f9eaa9c827d75d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:12:16.467145 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:16.467502 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:12:16.469066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1446/2910] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:12:16 2022] DELETE /identity/v3/projects/200b109c72ac43b7a7f9eaa9c827d75d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:12:21.555041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b747-be9f-4895-a790-1651d3a31732 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:21.555576 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b747-be9f-4895-a790-1651d3a31732 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:21.555700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b747-be9f-4895-a790-1651d3a31732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:21.572492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d4f7b747-be9f-4895-a790-1651d3a31732 None None] MFA Rules not processed for user `14fecc08cd364a48a8640d522577aa39`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 16:12:21.573691 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d4f7b747-be9f-4895-a790-1651d3a31732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:21.627010 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1465/2911] 10.222.0.98 () {58 vars in 957 bytes} [Fri Jul 15 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 16:12:21.938872 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:21.940292 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:22.007060 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254f23-a21b-47f6-9ac4-ce209d25f893 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:22.011155 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:22.011814 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:22.012100 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:22.012376 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:22.014066 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:22.023380 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:22.025680 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:22.034050 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1447/2912] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:24.241502 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b747-be9f-4895-a790-1651d3a31732 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:24.242961 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d4f7b747-be9f-4895-a790-1651d3a31732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:24.256149 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:12:24.312356 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b747-be9f-4895-a790-1651d3a31732 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:24.317407 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:24.318057 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:24.318345 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:24.320689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:24.320689 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:24.332801 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:24.334433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:24.344594 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1466/2913] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:32.030051 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=36,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:12:34.543739 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=27 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:12:41.650135 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:41.652236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:41.663437 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b3cb3-f7e7-437d-b9ba-5048bf2b8ea0 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:41.667396 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:41.668354 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:41.668780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:41.669245 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:41.671055 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:41.735215 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:41.736709 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:41.746268 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1448/2914] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:43.639019 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:12:43.640479 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:43.743158 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f023c-19cb-4b60-9d58-7ccfcc3e37d1 service placement] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:12:43.749296 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf670978cc2e4c529093a191bc26f885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:12:43.749952 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:12:43.750242 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:12:43.750522 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:12:43.752317 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:12:43.766018 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:12:43.767666 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:12:43.778790 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1467/2915] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:12:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:12:51.741968 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=30 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:12:53.773873 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=46 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:14:35.163676 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:14:35.175236 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e757a-d5d0-4bd3-ad62-3e21aa38d3cb service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:14:35.177194 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:14:35.177876 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:14:35.178171 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:14:35.178455 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:14:35.180352 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:14:35.190965 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:14:35.193162 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:14:35.202009 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1449/2916] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:14:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:14:45.197892 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=6 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:14:50.787764 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:14:50.797230 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d17a95-62f1-4912-9289-5aa20db509e0 service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:14:50.800845 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d20de9c921942d1afeb63e87272f782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:14:50.801476 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:14:50.801763 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:14:50.802041 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:14:50.803816 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:14:50.866235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:14:50.867714 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:14:50.877264 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1468/2917] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:14:58.143635 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:14:58.145103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:14:58.161617 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333440-7cde-4f09-90c7-94b5c374bdaa service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:14:58.167902 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:14:58.168785 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:14:58.169072 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:14:58.169348 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] PATH_INFO: `/v3/users/14fecc08cd364a48a8640d522577aa39` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:14:58.170560 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14fecc08cd364a48a8640d522577aa39)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:14:58.172527 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:14:58.384339 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14fecc08cd364a48a8640d522577aa39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:58.396103 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14fecc08cd364a48a8640d522577aa39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:58.396789 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14fecc08cd364a48a8640d522577aa39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:58.397392 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14fecc08cd364a48a8640d522577aa39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:58.407573 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14fecc08cd364a48a8640d522577aa39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:58.485329 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1450/2918] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:14:58 2022] DELETE /identity/v3/users/14fecc08cd364a48a8640d522577aa39 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:14:58.745738 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:14:58.758181 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde33193-9f96-40f5-be8e-1ebf3f61bc9e service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:14:58.820113 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:14:58.820969 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:14:58.821256 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:14:58.821534 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] PATH_INFO: `/v3/projects/801e210cf8c64e7184b178fbde47860c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:14:58.822700 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=801e210cf8c64e7184b178fbde47860c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:14:58.824678 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:14:58.926049 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801e210cf8c64e7184b178fbde47860c', 'request_id': 'req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:58.932434 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801e210cf8c64e7184b178fbde47860c', 'request_id': 'req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.009680 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801e210cf8c64e7184b178fbde47860c', 'request_id': 'req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.020945 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] The token cache is being invalidate because project 801e210cf8c64e7184b178fbde47860c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:14:59.021235 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.022435 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.024577 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1469/2919] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:14:58 2022] DELETE /identity/v3/projects/801e210cf8c64e7184b178fbde47860c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:14:59.028787 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:14:59.030249 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:14:59.050521 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125a322-5f71-458d-8dc1-e9e598d3a39e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:14:59.107181 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:14:59.108084 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:14:59.108374 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:14:59.108655 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] PATH_INFO: `/v3/users/b1f79c58c23e48ad89a8d05564fea50b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:14:59.109855 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f79c58c23e48ad89a8d05564fea50b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:14:59.111883 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:14:59.332235 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f79c58c23e48ad89a8d05564fea50b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.344497 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f79c58c23e48ad89a8d05564fea50b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.345199 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f79c58c23e48ad89a8d05564fea50b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.345865 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f79c58c23e48ad89a8d05564fea50b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.352780 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f79c58c23e48ad89a8d05564fea50b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.354441 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:14:59.415900 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1451/2920] 10.222.0.98 () {58 vars in 1223 bytes} [Fri Jul 15 16:14:59 2022] DELETE /identity/v3/users/b1f79c58c23e48ad89a8d05564fea50b => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:14:59.670420 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:14:59.673433 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:14:59.687720 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99f20c-ac74-45b1-8a0e-e9490a85cf0e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:14:59.708507 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:14:59.746137 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb07001f45bb40c8b8c84b7d67d1c304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c42afd51dd0d44aba511a26dccae48ce', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:14:59.747001 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:14:59.747334 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:14:59.747631 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] PATH_INFO: `/v3/projects/1ae5a68107b04f41ba4bc3284d7ba322` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:14:59.748812 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae5a68107b04f41ba4bc3284d7ba322)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:14:59.750716 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:14:59.864243 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae5a68107b04f41ba4bc3284d7ba322', 'request_id': 'req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.869727 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae5a68107b04f41ba4bc3284d7ba322', 'request_id': 'req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.925926 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae5a68107b04f41ba4bc3284d7ba322', 'request_id': 'req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.937145 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] The token cache is being invalidate because project 1ae5a68107b04f41ba4bc3284d7ba322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 16:14:59.937492 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.938160 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9da3abd1-70c7-4e4c-a5f8-630b91e9cb7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 16:14:59.939751 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1470/2921] 10.222.0.98 () {58 vars in 1232 bytes} [Fri Jul 15 16:14:59 2022] DELETE /identity/v3/projects/1ae5a68107b04f41ba4bc3284d7ba322 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 16:15:09.526166 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:15:09.983551 dv-cinder-835878-3-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 16:17:35.523137 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 16:17:35.524717 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:17:35.594257 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cd6ba-8cea-4ec7-b022-34593898dd74 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 16:17:35.652923 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3a084-7c8e-4384-8560-8285f6f780e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e57e2b49a8624266b12c78a78059cc80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ffc55a6490647cfa8f543334e168150', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 16:17:35.653570 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3a084-7c8e-4384-8560-8285f6f780e6 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 16:17:35.653858 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3a084-7c8e-4384-8560-8285f6f780e6 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 16:17:35.654136 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3a084-7c8e-4384-8560-8285f6f780e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 16:17:35.655948 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-acc3a084-7c8e-4384-8560-8285f6f780e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 16:17:35.667059 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3a084-7c8e-4384-8560-8285f6f780e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 16:17:35.668630 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3a084-7c8e-4384-8560-8285f6f780e6 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 16:17:35.678404 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1452/2922] 10.222.0.98 () {60 vars in 1365 bytes} [Fri Jul 15 16:17:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 16:17:45.674230 dv-cinder-835878-3-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=58 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}